Compare commits

..
2 Commits
17 changed files with 1224 additions and 1132 deletions
+12 -3
View File
@@ -1,4 +1,13 @@
# Generated firmware artifacts # Build output
/build/ /build/
tools/__pycache__/
**/__pycache__/ # Firmware images (allow tracked official stock baselines)
*.img
!Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img
!NEBULA_ota_img_V1.1.0.30.img
# Python cache & temporary files
__pycache__/
*.py[cod]
*$py.class
*.tmp
+36 -41
View File
@@ -1,50 +1,45 @@
# Firmware Changelog # Firmware Changelog
Differences between the stock Creality Ender-3 V3 KE image and our custom build. Differences, release milestones, and architecture changes across this workspace.
## Baseline ---
- Stock image: `Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img` ## 1.0.0 — OpenKE Stock USB Installer (2026-09-24)
- Custom build line: `DarKE_KE_F005_ota_img_V11034.img`
## Firmware Changes ### New Features & Architecture Transformation
* **Transformed Repo into OpenKE Installer**: Repurposed workspace from a stock pre-root patcher into a full-featured stock-compatible installer generator for **OpenKE** (Linux 6.6-rt + Klipper/Moonraker/GuppyScreen).
* **Zero-Disassembly / Single-USB Install**: Users can install OpenKE directly from an unmodified stock printer or Nebula Pad via the touchscreen UI without needing Ingenic mask-ROM recovery mode, cloner tools, or prior root access.
* **Dual Target Hardware Profiles**:
* **Ender-3 V3 KE** (Board target `F005`)
* **Creality Nebula Smart Kit / Nebula Pad** (Board target `NEBULA`)
* **Standalone Build Script (`build_openke_image.sh`)**:
* Automates preflight checks, asset selection, packaging, and validation.
* Supports `--target {ke,nebula,all}`, `--version`, `--artifacts`, and `--output`.
* **Direct OpenKE Packager (`tools/pack_openke.py`)**:
* Directly encapsulates production OpenKE `xImage` and `rootfs.squashfs` without requiring SquashFS re-compression.
* Validates hardware partition budgets: Kernel $\le$ 8 MiB (5.54 MiB used), Rootfs $\le$ 500 MiB (330.85 MiB used).
* Validates SHA256 checksums against `build-manifest.txt`.
* Generates 1 MiB payload chunks, MD5 hash sidecars, and Creality manifests.
* Implements pure-Python Unix MD5-crypt password derivation for any Creality board.
* Automatically verifies created 7z archives upon completion.
* **Cached RTOS Binaries**:
* Extracted and cached `assets/zero.bin` (KE, 432 KB) and `assets/zero_nebula.bin` (Nebula, 424 KB), making builds completely standalone without needing the 118 MB stock firmware images.
- Added `overlay_rootfs/etc/init.d/S05agree_root` to create `/usr/data/creality/userdata/user_agree_root` at boot. ### Forensics, Bug Fixes & Live Qualification
- Updated the version metadata used by the UI: * **Resolved OTA Extraction Failure on Stock Updater**:
- `overlay_rootfs/etc/ota_info` * *Root Cause*: Creality's `/etc/ota_bin/local_ota_update.sh` computes `OTA_UNZIP_FILE_NAME=${OTA_FILE_NAME%.img*}` and expects `ota_config.in` at `<TEMP_DIR>/<OTA_UNZIP_FILE_NAME>/ota_config.in`. When the archive file and internal folder names drifted, extraction aborted.
- `overlay_rootfs/usr/share/klipper/config/F005/printer.cfg` * *Fix*: Ensured root directory inside the 7z archive strictly matches the `.img` filename without `.img`.
- Current display version: `v1.1.0.33` * **Resolved Version Comparison Rejection**:
* *Root Cause*: Creality's `master-server` (`UpgradeManager.c`) parses versions after `ota_img_V` using regex `[0-9]{1,3}`. Raw integers like `11034` failed comparison against dotted stock versions.
* *Fix*: Formatted all release versions with standard dotted numbers (e.g. `1.1.0.34`), ensuring `master-server` and `local_ota_update.sh` recognize the update as newer than stock (`1.1.0.30` / `1.1.0.17`).
* **Live Qualification**:
* Successfully verified on real Creality Nebula Pad hardware via USB flash drive.
## Package Identity ---
- OTA package version: `11034` ## 0.0.1 — Pre-Rooted Stock Firmware (Legacy)
- UI/display version: `v1.1.0.33`
- The package still flashes through the stock USB updater flow.
## Build System Changes * Baseline: `Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img`
* Added `overlay_rootfs/etc/init.d/S05agree_root` to create `/usr/data/creality/userdata/user_agree_root` at boot.
- `tools/ke_firmware.py` now: * Enabled root access on stock CrealityOS.
- extracts the stock OTA archive * Provided `tools/ke_firmware.py` for SquashFS metadata-preserving rebuilds.
- unpacks the stock SquashFS rootfs
- applies the overlay
- rebuilds the rootfs
- repacks the archive
- Repacking preserves stock metadata:
- symlinks stay symlinks
- executable bits stay intact
- ownership and timestamps are preserved where applicable
- Added archive validation and stock-vs-rebuilt compare support.
- New `etc/init.d/*` overlay scripts are forced to pack as executable.
## Verified
- The rebuilt image matches stock rootfs metadata outside the overlay.
- The custom image rebuilds cleanly and flashes successfully.
- The root-agree file is created at boot by init, not baked in as a static overlay file.
# Changelog:
## 0.0.1
Pre-Rooted
+61 -69
View File
@@ -1,69 +1,61 @@
# Ender-3 V3 KE Firmware Notes # Firmware Architecture & Packaging Notes
## Stock firmware source This document contains technical reference information on stock firmware layouts, encryption passwords, partition budgets, and packaging mechanics for the **Ender-3 V3 KE** and **Creality Nebula Pad**.
- Downloaded official image: `Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img` ---
- Official version: `V1.1.0.17`
- Manifest layers inside the archive: ## 1. Stock Firmware Baselines
- `xImage` (`kernel`)
- `rootfs.squashfs` (`rootfs`) | Target | Official Baseline Image | Version | Board Name | Derived 7z Encryption Password |
- `zero.bin` (`rtos`) | :--- | :--- | :--- | :--- | :--- |
| **Ender-3 V3 KE** | `Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img` | `V1.1.0.17` | `F005` | `$1$cxswfile$ZFd0RWFYkJQugbtKVGL9y0` |
## 7z archive password | **Nebula Pad** | `NEBULA_ota_img_V1.1.0.30.img` | `V1.1.0.30` | `NEBULA` | `$1$cxswfile$XG7ANbYX3bq2H2xmmwrka.` |
The official Creality firmware archive is password-protected. ### Password Derivation Algorithm
```bash
Password: openssl passwd -1 -salt cxswfile "${BOARD_NAME}C3_7e_bz"
```
`$1$cxswfile$ZFd0RWFYkJQugbtKVGL9y0` Or in Python:
```python
## Workspace layout derive_creality_password(board_name) # implemented in tools/pack_openke.py
```
- Stock archive cache: `./Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img`
- Overlay tree for our changes: `./overlay_rootfs/` ---
- Build cache: `./build/_toolcache/squashfs-tools-ng-1.3.2-mingw64/`
- Temporary build workspace: whichever directory you pass with `--build-dir` ## 2. Partition Capacities & Budget Qualification
## Useful edit points in the stock rootfs Creality's X2000 partition table enforces hard upper bounds on flashable images:
- Klipper startup/service hook: `rootfs_extract/etc/init.d/S55klipper_service` | Sub-Payload | Target Partition | Hardware Limit | OpenKE Payload Size | Headroom |
- WebRTC startup/service hook: `rootfs_extract/etc/init.d/S97webrtc` | :--- | :--- | :--- | :--- | :--- |
- Stock printer profile: | **`xImage`** (Linux kernel) | `kernel` (`p5`) / `kernel2` (`p6`) | **8 MiB** (8,388,608 B) | **5.54 MiB** (5,541,952 B) | +2.71 MiB (66.1% used) |
- `rootfs_extract/usr/share/klipper/config/F005/printer.cfg` | **`rootfs.squashfs`** | `rootfs` (`p7`) / `rootfs2` (`p8`) | **500 MiB** (524,288,000 B) | **330.85 MiB** (346,923,008 B) | +169.15 MiB (66.2% used) |
- `rootfs_extract/usr/share/klipper/config/F005/gcode_macro.cfg` | **`zero.bin`** (RTOS) | `rtos` (`p3`) / `rtos2` (`p4`) | **4 MiB** (4,194,304 B) | **0.41–0.42 MiB** (~430 KiB) | +3.58 MiB (10.3% used) |
- `rootfs_extract/usr/share/klipper/config/F005/printer_params.cfg`
- Runtime printer config is copied into `/usr/data/printer_data/config/` on the device Both kernel and rootfs fit comfortably with substantial safety margins.
## Notes ---
- The shipped `.img` file is a 7z archive, not a raw flash image. ## 3. Tooling Overview
- The build flow starts from the stock archive, overlays `overlay_rootfs/`, and repacks the result.
- The helper uses native `rdsquashfs` and `gensquashfs` on Linux. Install * **[`build_openke_image.sh`](file:///root/Development/OpenKE/DarKE/build_openke_image.sh)**:
`squashfs-tools-ng` so those binaries are on `PATH`. On Windows it uses the Top-level script for building ready-to-flash OpenKE `.img` files directly from build artifacts.
bundled `rdsquashfs.exe` / `gensquashfs.exe` tools from `squashfs-tools-ng`. * **[`tools/pack_openke.py`](file:///root/Development/OpenKE/DarKE/tools/pack_openke.py)**:
- The repack path is metadata-preserving: it reuses the stock SquashFS layout so symlinks and execute bits survive the round trip. Core Python packager. Validates partition budgets, verifies SHA256 against `build-manifest.txt`, slices payloads into 1 MiB chunks with MD5 sidecars, derives board passwords, and compresses encrypted 7z envelopes.
- `xImage.full` is a stock U-Boot `uImage` kernel blob. Its raw payload can be derived if kernel-side edits are needed. * **[`assets/zero.bin`](file:///root/Development/OpenKE/DarKE/assets/zero.bin)**:
- Stock-updater-facing version scheme in this workspace: `11034` Cached RTOS binary for the Ender-3 V3 KE (`F005`).
- UI/display version stays on the dotted printer-side label `v1.1.0.33` * **[`assets/zero_nebula.bin`](file:///root/Development/OpenKE/DarKE/assets/zero_nebula.bin)**:
- The display/version string is sourced from `overlay_rootfs/etc/ota_info`. Cached RTOS binary for the Nebula Smart Kit / Nebula Pad (`NEBULA`).
- The firmware image is intended to flash through the stock USB updater without any manual printer-side file edits. * **[`tools/ke_firmware.py`](file:///root/Development/OpenKE/DarKE/tools/ke_firmware.py)**:
- The helper build process keeps the stock updater happy by using the numeric package version and stock archive layout. Legacy utility for extracting stock images, overlaying files onto stock rootfs, and repacking pre-rooted stock images.
## Rebuild ---
Use the helper script: ## 4. Key Rules for Creality OTA Packages
```powershell From reverse-engineering `/etc/ota_bin/local_ota_update.sh`:
python .\tools\ke_firmware.py build --build-dir D:\DarKE_build --stock-archive .\Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img --overlay-dir .\overlay_rootfs --output .\build\DarKE_KE_F005_ota_img_V11034.img
``` 1. **Folder Name Parity**: The root directory inside the 7z archive **must match the `.img` filename** without the `.img` extension.
- Example: `NEBULA_ota_img_V1.1.0.34.img` $\rightarrow$ folder inside 7z must be `NEBULA_ota_img_V1.1.0.34/`.
Compare the rebuilt rootfs against stock, allowing only the overlay tree to differ: 2. **Dotted Versioning**: The version must be formatted with dots (e.g. `1.1.0.34`), not a raw integer, so `master-server`'s `[0-9]{1,3}` parser can compare versions.
3. **Chunking**: Chunks are 1,048,576 bytes each. Chunk `0000` points to the whole-file MD5, and subsequent chunks chain the previous chunk's MD5.
```powershell
python .\tools\ke_firmware.py compare --stock-archive .\Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img --rebuilt-archive .\build\DarKE_KE_F005_ota_img_V11034.img --overlay-dir .\overlay_rootfs
```
## Stock-safe flash summary
- Put the generated `DarKE_KE_F005_ota_img_V11034.img` on a USB drive.
- Flash it with the printer's built-in updater.
Binary file not shown.
+138 -41
View File
@@ -1,49 +1,146 @@
# Ender-3 V3 KE firmware workspace # OpenKE Stock USB Installer & Firmware Workspace
This repo contains a stock KE firmware image, a small overlay tree of our changes, and a helper script for rebuilding an `.img` package.
## Files
- `FIRMWARE_NOTES.md`: archive password and firmware layout notes A standalone utility to package **[OpenKE](https://github.com/coreflake1/OpenKE)** (Linux 6.6-rt + Klipper/Moonraker/GuppyScreen) directly into stock CrealityOS-supported OTA update (`.img`) containers.
- `tools/ke_firmware.py`: extract/rebuild helper
- `requirements.txt`: Python dependency list for the helper script
## Python setup This enables a **zero-disassembly, single-USB installation of OpenKE straight from an unmodified stock printer or Nebula Pad touchscreen** without requiring Ingenic mask-ROM recovery mode, cloner utilities, or prior root access.
Install the helper dependency with: ---
## Supported Hardware
| Device | Board Identifier | Official Stock Baseline | Generated OTA Installer Package |
| :--- | :--- | :--- | :--- |
| **Creality Ender-3 V3 KE** | `F005` | `Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img` | [`build/Ender-3_V3_KE_F005_ota_img_V1.1.0.34.img`](build/Ender-3_V3_KE_F005_ota_img_V1.1.0.34.img) |
| **Creality Nebula Pad** | `NEBULA` | `NEBULA_ota_img_V1.1.0.30.img` | [`build/NEBULA_ota_img_V1.1.0.34.img`](build/NEBULA_ota_img_V1.1.0.34.img) |
---
## Table of Contents
1. [Quick Start](#1-quick-start)
2. [Flashing Guide](#2-flashing-guide)
3. [Safety & Dual-Slot A/B Architecture](#3-safety--dual-slot-ab-architecture)
4. [Technical Documentation Index](#4-technical-documentation-index)
5. [Repository Structure](#5-repository-structure)
---
## 1. Quick Start
### Prerequisites
* Linux, macOS, or Windows
* Python 3.8+
* `7z` (p7zip) or Python `py7zr` (`pip install -r requirements.txt`)
### Generate Installer Images
Build for **Ender-3 V3 KE** (default):
```bash ```bash
pip install -r requirements.txt ./build_openke_image.sh --target ke
``` ```
On Linux, install `squashfs-tools-ng` so `gensquashfs` and `rdsquashfs` are Build for **Nebula Pad**:
available on `PATH`. On Windows, the helper uses the bundled `.exe` tools. ```bash
./build_openke_image.sh --target nebula
```
## Rebuild flow Build for **Both Devices in One Run**:
```bash
```powershell ./build_openke_image.sh --target all
python .\tools\ke_firmware.py build --stock-archive .\Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img --overlay-dir .\overlay_rootfs --output .\build\DarKE_KE_F005_ota_img_V11034.img ```
```
By default, the script automatically:
If the stock archive is missing, pass `--stock-url` and the helper will download it before extracting. 1. Locates OpenKE build artifacts in `../OpenKE/artifacts/buildroot-halley5-v30-image/`.
If your `C:` drive is tight, point `--build-dir` at a larger drive so the stock rootfs can be unpacked there. 2. Validates partition budgets:
- `xImage`: 5.54 MiB / 8 MiB (66.1% capacity, 2.71 MiB free).
The build step downloads or reuses the stock archive, extracts it, overlays `overlay_rootfs/`, rebuilds the root filesystem with the Windows `squashfs-tools-ng` binaries, and repacks the Creality image envelope with `py7zr`. - `rootfs.squashfs`: 330.85 MiB / 500 MiB (66.2% capacity, 169.15 MiB free).
The rebuild is metadata-preserving, so stock symlinks and executable bits survive the round trip. 3. Verifies SHA256 checksums against `build-manifest.txt` (if present).
4. Slices payloads into 1 MiB chunks with Creality MD5 sidecars.
## Compare flow 5. Encrypts the 7z container with the board-derived Creality key (`$1$cxswfile$...`).
6. Verifies archive integrity and outputs the `.img` to `build/`.
```powershell
python .\tools\ke_firmware.py compare --stock-archive .\Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img --rebuilt-archive .\build\DarKE_KE_F005_ota_img_V11034.img --overlay-dir .\overlay_rootfs ### Custom Arguments & CLI Flags
```
```bash
Use this after a build to confirm the rebuilt rootfs still matches stock everywhere except the overlayed files. # Custom artifacts directory and custom version
./build_openke_image.sh --target nebula --artifacts /path/to/artifacts --version 1.1.0.35
## Stock flash flow
# Or invoke the Python packager directly
1. Copy the generated `.img` to a USB drive. python3 tools/pack_openke.py --target nebula --artifacts-dir ../OpenKE/artifacts/buildroot-halley5-v30-image
2. Insert the USB drive into the Ender-3 V3 KE. ```
3. Use the built-in updater to flash the image.
4. After reboot, the firmware reports the UI/display version from the package metadata. ---
This flow does not require manual edits on the printer itself. ## 2. Flashing Guide
1. Format a USB flash drive as **FAT32** with an **MBR** (Master Boot Record) partition table.
2. Copy the generated `.img` file to the **root** of the USB drive:
- For Ender-3 V3 KE: `Ender-3_V3_KE_F005_ota_img_V1.1.0.34.img`
- For Nebula Pad: `NEBULA_ota_img_V1.1.0.34.img`
3. Insert the USB drive into the front USB port of the printer / Nebula Pad.
4. On the touchscreen: Navigate to **Settings** $\rightarrow$ **Firmware Update** $\rightarrow$ **Local Update**.
5. Select the firmware image and confirm the update.
6. The stock updater will:
- Flash `xImage` to `kernel2` (`/dev/mmcblk0p6`).
- Flash `rootfs.squashfs` to `rootfs2` (`/dev/mmcblk0p8`).
- Flash `zero.bin` to `rtos2` (`/dev/mmcblk0p4`).
- Set the bootloader marker in `/dev/mmcblk0p1` to `ota:kernel2`.
- Automatically reboot into OpenKE.
7. On first boot, OpenKE seeds `/usr/data/openke/` and starts Klipper, Moonraker, and GuppyScreen.
---
## 3. Safety & Dual-Slot A/B Architecture
* **Non-Destructive Install**: The stock CrealityOS operating system resides permanently in **Slot 1** (`mmcblk0p5`/`p7`) and is **never overwritten**. OpenKE is installed cleanly into **Slot 2** (`mmcblk0p6`/`p8`).
* **Persistent Data Isolation**: Both operating systems share `/usr/data`, but OpenKE isolates its entire runtime in `/usr/data/openke/`. Stock Creality configs (`/usr/data/creality/`), printer calibrations, and WiFi credentials remain completely untouched.
* **Instant Reversion to Stock**: If you ever want to revert to stock CrealityOS, simply toggle the boot marker back to Slot 1:
```bash
# From OpenKE SSH shell:
echo -n "ota:kernel" > /dev/mmcblk0p1 && sync && reboot
```
---
## 4. Technical Documentation Index
For in-depth architectural and developer documentation:
* **[Creality OTA Packaging Specification](docs/CREALITY_OTA_SPECIFICATION.md)**:
Full reverse-engineered specification of Creality's 7z container, MD5-crypt key derivation, 1 MiB payload chunking, manifest formats, and updater execution flow.
* **[Hardware A/B Partition Model](docs/A_B_PARTITION_MODEL.md)**:
Physical partition table, partition capacities, bootloader marker mechanics, and `/usr/data` namespace isolation.
* **[Troubleshooting & Diagnostics](docs/TROUBLESHOOTING.md)**:
Common USB update pitfalls, drive formatting, version string parsing, and how to inspect live updater logs on the device.
* **[Firmware Notes](FIRMWARE_NOTES.md)**:
Official baselines, derived encryption keys, partition budgets, and tool usage notes.
* **[Firmware Changelog](FIRMWARE_CHANGELOG.md)**:
Project history from DarKE v0.0.1 pre-root to OpenKE USB Installer v1.0.0.
---
## 5. Repository Structure
```
DarKE/
├── Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img # Stock CrealityOS baseline for Ender-3 V3 KE (F005)
├── NEBULA_ota_img_V1.1.0.30.img # Stock CrealityOS baseline for Nebula Pad (NEBULA)
├── build_openke_image.sh # Automated build wrapper script
├── README.md # Workspace overview & quickstart
├── FIRMWARE_NOTES.md # Architecture notes & encryption keys
├── FIRMWARE_CHANGELOG.md # Version changelog & release milestones
├── requirements.txt # Python dependencies (`py7zr`)
├── assets/ # Standalone RTOS binaries
│ ├── zero.bin # Stock RTOS for Ender-3 V3 KE (F005)
│ └── zero_nebula.bin # Stock RTOS for Nebula Pad (NEBULA)
├── docs/ # In-depth technical documentation
│ ├── CREALITY_OTA_SPECIFICATION.md # Creality OTA container & updater internals
│ ├── A_B_PARTITION_MODEL.md # GPT partition layout & slot mechanics
│ └── TROUBLESHOOTING.md # USB diagnostics & common issues
├── tools/
│ └── pack_openke.py # OpenKE stock OTA packager
└── build/ # Generated OTA installer images
├── Ender-3_V3_KE_F005_ota_img_V1.1.0.34.img
└── NEBULA_ota_img_V1.1.0.34.img
```
BIN
View File
Binary file not shown.
Binary file not shown.
+147
View File
@@ -0,0 +1,147 @@
#!/usr/bin/env bash
#
# build_openke_image.sh
#
# Generates a stock-compatible CrealityOS OTA firmware (.img) directly from
# OpenKE build artifacts. The resulting file can be placed on a FAT32 USB drive
# and flashed via the stock touchscreen UI to install OpenKE to Slot 2.
#
# Supports:
# - Ender-3 V3 KE (Target F005)
# - Nebula Smart Kit / Nebula Pad (Target NEBULA)
#
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "${SCRIPT_DIR}"
# Defaults
DEFAULT_ARTIFACTS="${SCRIPT_DIR}/../OpenKE/artifacts/buildroot-halley5-v30-image"
DEFAULT_VERSION="1.1.0.34"
DEFAULT_TARGET="ke"
ARTIFACTS_DIR="${DEFAULT_ARTIFACTS}"
VERSION="${DEFAULT_VERSION}"
TARGET="${DEFAULT_TARGET}"
OUTPUT=""
usage() {
cat << EOF
Usage: $(basename "$0") [OPTIONS]
Packages OpenKE kernel and rootfs into a stock CrealityOS-supported update .img.
Options:
-t, --target TARGET Target hardware profile:
ke - Ender-3 V3 KE (F005) [Default]
nebula - Creality Nebula Pad (NEBULA)
all - Build images for both devices
-a, --artifacts DIR Directory containing OpenKE build artifacts (xImage, rootfs.squashfs)
(Default: ../OpenKE/artifacts/buildroot-halley5-v30-image)
-v, --version VER Package version for Creality updater (Default: 1.1.0.34)
-o, --output FILE Output .img file path (single target only)
-h, --help Show this help message and exit
Examples:
./build_openke_image.sh # Builds for Ender-3 V3 KE
./build_openke_image.sh --target nebula # Builds for Nebula Pad
./build_openke_image.sh --target all # Builds both KE and Nebula images
EOF
exit 0
}
while [[ $# -gt 0 ]]; do
case "$1" in
-t|--target)
TARGET="$2"
shift 2
;;
-a|--artifacts)
ARTIFACTS_DIR="$2"
shift 2
;;
-v|--version)
VERSION="$2"
shift 2
;;
-o|--output)
OUTPUT="$2"
shift 2
;;
-h|--help)
usage
;;
*)
echo "Unknown option: $1" >&2
usage
;;
esac
done
echo "=================================================================="
echo " OpenKE Stock OTA Firmware Generator"
echo "=================================================================="
echo " Target Device: ${TARGET}"
echo " Artifacts Directory: ${ARTIFACTS_DIR}"
echo " Package Version: ${VERSION}"
if [[ -n "${OUTPUT}" ]]; then
echo " Custom Output: ${OUTPUT}"
fi
echo "=================================================================="
# Check Python3
if ! command -v python3 &>/dev/null; then
echo "ERROR: python3 is not installed or not in PATH." >&2
exit 1
fi
# Ensure assets are present
mkdir -p "${SCRIPT_DIR}/assets"
if [[ ! -f "${SCRIPT_DIR}/assets/zero.bin" && -f "${SCRIPT_DIR}/Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img" ]]; then
echo "[*] Extracting assets/zero.bin from KE stock image..."
7z e -p'$1$cxswfile$ZFd0RWFYkJQugbtKVGL9y0' -r "${SCRIPT_DIR}/Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img" 'zero.bin*' -o"${SCRIPT_DIR}/assets" -y >/dev/null
cp "${SCRIPT_DIR}/assets/zero.bin.0000."* "${SCRIPT_DIR}/assets/zero.bin" 2>/dev/null || true
rm -f "${SCRIPT_DIR}/assets/zero.bin.0000."*
fi
if [[ ! -f "${SCRIPT_DIR}/assets/zero_nebula.bin" && -f "${SCRIPT_DIR}/NEBULA_ota_img_V1.1.0.30.img" ]]; then
echo "[*] Extracting assets/zero_nebula.bin from Nebula stock image..."
7z e -p'$1$cxswfile$XG7ANbYX3bq2H2xmmwrka.' -r "${SCRIPT_DIR}/NEBULA_ota_img_V1.1.0.30.img" 'zero.bin*' -o"${SCRIPT_DIR}/assets" -y >/dev/null
cp "${SCRIPT_DIR}/assets/zero.bin.0000."* "${SCRIPT_DIR}/assets/zero_nebula.bin" 2>/dev/null || true
rm -f "${SCRIPT_DIR}/assets/zero.bin.0000."*
fi
# Check OpenKE artifacts
if [[ ! -d "${ARTIFACTS_DIR}" ]]; then
echo "ERROR: Artifacts directory not found: ${ARTIFACTS_DIR}" >&2
echo "Please build OpenKE first or specify --artifacts <dir>." >&2
exit 1
fi
if [[ ! -f "${ARTIFACTS_DIR}/xImage" ]]; then
echo "ERROR: ${ARTIFACTS_DIR}/xImage does not exist." >&2
exit 1
fi
if [[ ! -f "${ARTIFACTS_DIR}/rootfs.squashfs" ]]; then
echo "ERROR: ${ARTIFACTS_DIR}/rootfs.squashfs does not exist." >&2
exit 1
fi
# Build arguments
ARGS=(
"--target" "${TARGET}"
"--artifacts-dir" "${ARTIFACTS_DIR}"
"--version" "${VERSION}"
)
if [[ -n "${OUTPUT}" ]]; then
ARGS+=("--output" "${OUTPUT}")
fi
python3 "${SCRIPT_DIR}/tools/pack_openke.py" "${ARGS[@]}"
echo
echo "Build complete! Output files are located in build/"
ls -lh "${SCRIPT_DIR}/build/"
+86
View File
@@ -0,0 +1,86 @@
# Hardware Partition Layout & Dual-Slot A/B Boot Model
This document details the eMMC physical partition layout, the hardware A/B slot mechanics, and the persistent storage architecture used by Creality X2000 devices (Ender-3 V3 KE and Nebula Pad).
---
## 1. Physical Partition Table
The onboard eMMC is partitioned with a standard GPT partition table:
| Partition | Label | Fixed Capacity | Mount / Role | Active Slot 1 (Stock) | Active Slot 2 (OpenKE) |
| :--- | :--- | :--- | :--- | :--- | :--- |
| `/dev/mmcblk0p1` | `ota` | **1 MiB** | Boot marker storage | Stores `ota:kernel` or `ota:kernel2` | Shared |
| `/dev/mmcblk0p2` | `sn_mac` | **1 MiB** | Factory serial & MAC addresses | Read-only identity | Shared |
| `/dev/mmcblk0p3` | `rtos` | **4 MiB** | Aux RTOS firmware (Slot 1) | Active | Inactive |
| `/dev/mmcblk0p4` | `rtos2` | **4 MiB** | Aux RTOS firmware (Slot 2) | Inactive | Active |
| `/dev/mmcblk0p5` | `kernel` | **8 MiB** | Linux uImage (Slot 1) | Active | Inactive |
| `/dev/mmcblk0p6` | `kernel2` | **8 MiB** | Linux uImage (Slot 2) | Inactive | Active |
| `/dev/mmcblk0p7` | `rootfs` | **500 MiB** | SquashFS rootfs (Slot 1) | Active | Inactive |
| `/dev/mmcblk0p8` | `rootfs2` | **500 MiB** | SquashFS rootfs (Slot 2) | Inactive | Active |
| `/dev/mmcblk0p9` | `rootfs_data` | **300 MiB** | Ext4 writable overlay | `/overlay` | `/overlay` |
| `/dev/mmcblk0p10` | `userdata` | **~6.1 GiB** | Ext4 persistent storage | Mounted at `/usr/data` | Mounted at `/usr/data` |
---
## 2. Boot Selection via the OTA Marker
The bootloader (U-Boot) reads the 1 MiB `ota` partition (`/dev/mmcblk0p1`) on power-up to decide which kernel and rootfs to mount. The partition contains a simple ASCII string:
* `ota:kernel` $\rightarrow$ Boots **Slot 1** (Stock CrealityOS: kernel `p5`, rootfs `p7`).
* `ota:kernel2` $\rightarrow$ Boots **Slot 2** (Custom / OpenKE: kernel `p6`, rootfs `p8`).
### How Stock OTA Updater Flashes to Slot 2
When you run a USB update from the stock Creality touchscreen:
1. Stock firmware queries which slot is currently running (`local_get_kernel_dev_path`).
2. Since the printer is running on Slot 1 (`ota:kernel`), the updater targets the **inactive** slot:
- Flashes `xImage` to `kernel2` (`mmcblk0p6`).
- Flashes `rootfs.squashfs` to `rootfs2` (`mmcblk0p8`).
- Flashes `zero.bin` to `rtos2` (`mmcblk0p4`).
3. Calls `local_set_next_boot_device`, which writes `ota:kernel2` into `mmcblk0p1`.
4. Reboots into OpenKE.
> [!NOTE]
> Stock firmware in Slot 1 (`mmcblk0p5` and `mmcblk0p7`) is **never overwritten** by this process.
---
## 3. Persistent Data Isolation (`/usr/data`)
`/dev/mmcblk0p10` is the single shared ~6.1 GiB persistent partition mounted at `/usr/data`. Both Slot 1 and Slot 2 share this partition, but they use strict directory separation:
```
/usr/data/
├── creality/ # Stock CrealityOS files (gcode, logs, userdata, factory data)
│ ├── userdata/
│ └── printer_data/
└── openke/ # OpenKE isolated runtime
├── apps/ # Mutable checkouts (klipper, moonraker, mainsail)
├── envs/ # Python virtual environments
├── system/ # Boot logs, activation state, machine configs
└── printer_data/ # Klipper config (printer.cfg), macros, Moonraker DB
```
Because OpenKE namespaces all its data under `/usr/data/openke/`, flashing OpenKE does not erase or corrupt stock print history, WiFi credentials, or factory calibration files.
---
## 4. Reverting to Stock Firmware
If you ever wish to return to stock CrealityOS, you do not need to reflash. You simply toggle the boot marker back to Slot 1:
### From OpenKE Shell (SSH)
```bash
echo -n "ota:kernel" > /dev/mmcblk0p1
sync
reboot
```
### From Stock CrealityOS (if toggling forward again)
```bash
. /etc/ota_bin/ota_utils.sh
. /etc/ota_bin/ota_local_method.sh
local_set_next_boot_device
reboot
```
+168
View File
@@ -0,0 +1,168 @@
# CrealityOS OTA Packaging & Update Subsystem Specification
This document provides a technical specification of Creality's Over-The-Air (OTA) firmware update container format, encryption model, payload chunking, and the on-device update execution engine used across the **Ender-3 V3 KE** (board target `F005`) and the **Nebula Smart Kit / Nebula Pad** (board target `NEBULA`).
---
## 1. Overview & Security Model
Creality's local (USB) and network OTA update mechanism on Ingenic X2000 devices is **integrity-checked but unauthenticated**:
* **Archive Envelope**: A password-protected 7z archive with encrypted headers (`-mhe=on`).
* **Encryption Key**: Derived deterministically from the hardware board name using standard Unix MD5 crypt.
* **Payload Verification**: Each sub-payload (`xImage`, `rootfs.squashfs`, `zero.bin`) is checked using MD5 checksums.
* **No Cryptographic Signatures**: The device does **not** enforce RSA digital signatures or public key verification during OTA updates. Unsigned custom kernels and root filesystems flash and boot cleanly through stock firmware.
---
## 2. Key Derivation Formula
The 7z archive password is generated per board type using standard Unix MD5 password encryption (`$1$` format) with a hardcoded salt:
$$\text{Password} = \text{MD5-Crypt}(\text{Key} = \text{BOARD\_NAME} + \text{"C3\_7e\_bz"}, \text{Salt} = \text{"cxswfile"})$$
In shell (using `openssl`):
```bash
openssl passwd -1 -salt cxswfile "${BOARD_NAME}C3_7e_bz"
```
### Known Board Encryption Keys
| Target Device | Board Identifier | Input String | Resulting 7z Password |
| :--- | :--- | :--- | :--- |
| **Ender-3 V3 KE** | `F005` | `F005C3_7e_bz` | `$1$cxswfile$ZFd0RWFYkJQugbtKVGL9y0` |
| **Nebula Pad** | `NEBULA` | `NEBULAC3_7e_bz` | `$1$cxswfile$XG7ANbYX3bq2H2xmmwrka.` |
| **Creality K1** | `CR4CU220812S12` | `CR4CU220812S12C3_7e_bz` | Board-derived key |
> [!TIP]
> `tools/pack_openke.py` includes a pure-Python implementation of Unix MD5-crypt, making password derivation completely cross-platform without external tools.
---
## 3. Archive Structure & Layout
Creality's update extractor (`/etc/ota_bin/local_ota_update.sh`) has strict structural expectations.
### Directory Hierarchy
For an archive file named `<ARCHIVE_STEM>.img` with package version `<VERSION>` (e.g. `NEBULA_ota_img_V1.1.0.34.img` with version `1.1.0.34`):
```
<ARCHIVE_STEM>/ # Top-level directory inside 7z
├── ota_config.in # Root version pointer
└── ota_v<VERSION>/ # Versioned payload folder
├── ota_v<VERSION>.ok # Readiness marker (touch file)
├── ota_update.in # Primary payload manifest
├── ota_md5_xImage.<FULL_MD5> # Chunk MD5 sidecar for kernel
├── ota_md5_rootfs.squashfs.<FULL_MD5> # Chunk MD5 sidecar for rootfs
├── ota_md5_zero.bin.<FULL_MD5> # Chunk MD5 sidecar for RTOS
├── xImage.0000.<FULL_MD5> # 1 MiB chunk 0
├── xImage.0001.<CHUNK0_MD5> # 1 MiB chunk 1
├── ...
├── rootfs.squashfs.0000.<FULL_MD5>
├── ...
└── zero.bin.0000.<FULL_MD5>
```
---
## 4. Manifest Formats
### 1. `ota_config.in`
Located at the root of the extracted folder:
```ini
current_version=1.1.0.34
```
### 2. `ota_v<VERSION>.ok`
An empty 0-byte or 1-byte newline file placed in `ota_v<VERSION>/`. If missing, the updater aborts.
### 3. `ota_update.in`
Lists each payload with its partition type, internal filename, byte size, and full-file MD5:
```ini
ota_version=1.1.0.34
img_type=kernel
img_name=xImage
img_size=5541952
img_md5=f3bf93ad2fd1e770dacb3062ba9cf05c
img_type=rootfs
img_name=rootfs.squashfs
img_size=346923008
img_md5=7d0d3742dedd7a236632e4f7439f52dd
img_type=rtos
img_name=zero.bin
img_size=424264
img_md5=8e852ae9b8f5b78f3818e4236c3e8f1d
```
### 4. Sidecar Checksum Files (`ota_md5_<NAME>.<FULL_MD5>`)
Text files containing the MD5 checksum of each 1 MiB chunk in order, separated by newlines:
```
e10adc3949ba59abbe56e057f20f883e
c33367701511b4f6020ec61ded352059
...
```
---
## 5. Payload Slicing & Chunking Algorithm
Creality splits large image files into fixed **1 MiB** (1,048,576 bytes) chunks to manage RAM constraints on the 208MB-RAM device during download and flashing:
1. **Chunk Naming Convention**:
$$\text{chunk\_filename} = \text{base\_name} + \text{"."} + \text{index:04d} + \text{"."} + \text{prev\_md5}$$
2. **First Chunk (`index = 0000`)**:
`prev_md5` is set to the **FULL MD5** of the unchunked source file.
3. **Subsequent Chunks (`index > 0000`)**:
`prev_md5` is the MD5 checksum of the **immediately preceding chunk** (`index - 1`).
4. **Final Chunk**:
Carries the remaining bytes ($\le 1,048,576$ bytes).
---
## 6. On-Device Update Execution Flow
Understanding the device-side execution path in `/etc/ota_bin/local_ota_update.sh` explains the strict naming rules:
```mermaid
sequenceDiagram
autonumber
actor User as User via Touchscreen
participant MS as master-server (UI)
participant LU as local_ota_update.sh
participant OF as ota_file (7z Decryptor)
participant BL as Block Devices (mmcblk0)
User->>MS: Selects USB Update
MS->>MS: Scans for 'ota_img_V' on USB
MS->>MS: Compares versions (regex [0-9]{1,3})
MS->>LU: Calls local_ota_update.sh /media/sda1/<NAME>.img
LU->>LU: Computes OTA_UNZIP_FILE_NAME=${OTA_FILE_NAME%.img*}
LU->>OF: Calls ota_file e <NAME>.img <TEMP_DIR>
OF->>OF: Derives key & extracts 7z archive
LU->>LU: Checks <TEMP_DIR>/<OTA_UNZIP_FILE_NAME>/ota_config.in
LU->>LU: Validates partition sizes against device nodes
LU->>BL: Streams xImage chunks -> mmcblk0p6 (kernel2)
LU->>BL: Streams rootfs chunks -> mmcblk0p8 (rootfs2)
LU->>BL: Streams zero.bin chunks -> mmcblk0p4 (rtos2)
LU->>BL: Writes "ota:kernel2" to /dev/mmcblk0p1
LU->>MS: Returns success
MS->>User: Reboots printer into OpenKE
```
### The Critical Folder-Name Matching Rule
Lines 312–320 of `local_ota_update.sh`:
```bash
OTA_FILE_NAME=$(basename ${OTA_FILE})
OTA_UNZIP_FILE_NAME=${OTA_FILE_NAME%.img*}
ota_site=${OTA_FILE_PATH}/${OTA_UNZIP_FILE_NAME}
/usr/bin/ota_file e ${OTA_FILE_PATH}/${OTA_FILE_NAME} ${OTA_FILE_PATH}
ota_site_config=$ota_site/ota_config.in
```
> [!IMPORTANT]
> If the archive file is named `NEBULA_ota_img_V1.1.0.34.img`, `OTA_UNZIP_FILE_NAME` evaluates to `NEBULA_ota_img_V1.1.0.34`. The root directory inside the 7z archive **must match that exact string**. If the archive has a different folder name (e.g. `OpenKE_...`), `cp -f $ota_site_config` fails and the update aborts immediately.
+71
View File
@@ -0,0 +1,71 @@
# Troubleshooting & USB Flashing Diagnostics
This document covers common issues when flashing OpenKE via USB OTA, root causes, and diagnostic steps.
---
## 1. "No firmware found" / "No local update available"
When you insert the USB drive and navigate to **Settings** $\rightarrow$ **Firmware Update** $\rightarrow$ **Local Update**, the screen reports no file found.
### Root Causes & Fixes:
1. **USB Drive File System**:
* The Creality Linux kernel does not reliably mount exFAT or NTFS drives on early boot.
* **Fix**: Format the USB flash drive as **FAT32** with an **MBR** (Master Boot Record) partition table. Avoid GPT partition tables on the flash drive.
2. **File Location**:
* The `.img` file must be placed in the **root directory** of the USB drive (e.g. `E:\NEBULA_ota_img_V1.1.0.34.img`), not inside a folder.
3. **Filename Format**:
* Creality's `master-server` scans for files matching `ota_img_V`.
* Files must be named:
- For Nebula Pad: `NEBULA_ota_img_V<version>.img`
- For Ender-3 V3 KE: `Ender-3_V3_KE_F005_ota_img_V<version>.img`
---
## 2. "Update failed" Immediately After Confirming
The screen detects the update, you click "Update Now", the progress bar appears for a split second, and it immediately reports "Update Failed".
### Root Cause:
* Creality's updater extracts the 7z archive to a temporary directory and looks for `ota_config.in` in `<TEMP_DIR>/<UNZIP_FOLDER>/ota_config.in`.
* If the root directory inside the 7z archive does not match the `.img` filename (without `.img`), `cp -f $ota_site_config` fails.
* **Resolution**: The updated `build_openke_image.sh` / `tools/pack_openke.py` guarantees this alignment automatically. Ensure you use images built with the latest packager.
---
## 3. "Current version is newest" / Won't Allow Flashing
The UI says the firmware is already up-to-date or newer.
### Root Cause:
* In `local_ota_update.sh`:
```bash
if [ $current_version -ge $ota_current_version ]; then
echo "ota not update: current version is newest"
exit 2
fi
```
* If your device is running `1.1.0.30` and the package version is `1.1.0.30` or lower, it refuses to flash.
* **Fix**: Pass a higher version number when running the build script:
```bash
./build_openke_image.sh --target nebula --version 1.1.0.35
```
---
## 4. How to Inspect Live Update Logs on the Printer
If you have SSH access to stock firmware and want to see what `local_ota_update.sh` is doing during a flash:
1. Connect to the printer via SSH:
```bash
ssh root@<printer-ip>
```
2. Trigger the local update manually from the command line to see stdout/stderr in real time:
```bash
sh -x /etc/ota_bin/local_ota_update.sh /media/sda1/NEBULA_ota_img_V1.1.0.34.img
```
3. Check temporary updater logs:
```bash
ls -la /usr/data/creality/ota_updater*
```
-1
View File
@@ -1 +0,0 @@
etc/init.d/S98sync_version
-12
View File
@@ -1,12 +0,0 @@
#!/bin/sh
AGREE_ROOT_FILE=/usr/data/creality/userdata/user_agree_root
case "$1" in
start)
mkdir -p /usr/data/creality/userdata
: > "$AGREE_ROOT_FILE"
;;
esac
true
-4
View File
@@ -1,4 +0,0 @@
ota_version=1.1.0.33
ota_board_name=F005
ota_compile_time=2024 08.06 09:12:56
ota_site=http://192.168.43.52/ota/board_test
@@ -1,256 +0,0 @@
# Ender-3V3KE
# Printer_size: 220x220x240
# Version: v1.1.0.33
# CreateDate: 2024/3/27
# mcu: chip: GD32F303RET6
# version: CR4NS200323C10
[include sensorless.cfg]
[include gcode_macro.cfg]
[include printer_params.cfg]
[mcu]
serial:/dev/ttyS1
baud:230400
restart_method: command
[force_move]
enable_force_move: True
[mcu rpi]
serial: /tmp/klipper_host_mcu
[bl24c16f]
i2c_mcu: rpi
i2c_bus: i2c.2
i2c_speed: 400000
######################################################
[prtouch_v2]
pres_cnt: 1 #探点次数
pres0_clk_pins: PA4 #压力检测时钟引脚配置
pres0_sdo_pins: PC6 #压力检测数据引脚配置
step_swap_pin: PA15
pres_swap_pin: PA15
step_base:2
# show_msg: True
tri_min_hold: 1000
tri_max_hold: 1500 #压力检测信息展示
speed: 1
# tri_wave_ip: 172.22.30.204
#####################################################
[z_compensate]
tri_min_hold: 1400
tri_max_hold: 2000 #压力检测信息展示
tri_expand_mm = 0.10
# tri_min_hold: 3
speed: 5
hot_start_temp: 180#擦喷嘴是最小温度
hot_rub_temp: 200#擦喷嘴是最小温度
hot_end_temp: 140#擦喷嘴是最小温度
bed_add_temp: 60#调平时的热床温度
clr_noz_start_x: -3 #擦喷头区域的起始x坐标(默认在热床后方正中心位置)
clr_noz_start_y: 20 #擦喷头区域的起始y坐标
clr_noz_len_x: 3 #擦喷头区域的x方向的长度
clr_noz_len_y: 50 #擦喷头区域的y方向的长度
pa_clr_dis_mm_x = 0
pa_clr_dis_mm_y =30
# show_msg = True
bl_offset: 0,27
noz_pos_center: 20,25
noz_pos_offset: 3,7
pumpback_mm: 10
vs_start_z_pos: 3
pr_probe_cnt: 3
pr_clear_probe_cnt: 3
type_nozz = 0
[printer]
kinematics: cartesian
max_velocity: 500
max_accel: 8000
max_accel_to_decel: 2500
max_z_velocity: 30
square_corner_velocity: 5.0
max_z_accel: 300
[idle_timeout]
timeout: 99999999
[stepper_x]
step_pin: PC2
dir_pin: !PB9
enable_pin: !PC3
microsteps: 16
rotation_distance: 40
endstop_pin: !PA5
position_endstop: -12
position_min: -12
position_max: 221
homing_speed: 30
homing_retract_dist:0 #10
[tmc2208 stepper_x]
uart_pin:PB12
interpolate: True
run_current:0.75
sense_resistor: 0.150
stealthchop_threshold: 0
[stepper_y]
step_pin: PB8
dir_pin: PB7
enable_pin: !PC3
microsteps: 16
rotation_distance: 60
endstop_pin: !PA6
position_endstop: -20
position_min: -20
position_max: 223
homing_speed: 30
homing_retract_dist:0
[tmc2208 stepper_y]
uart_pin:PB13
interpolate: True
run_current:0.75
sense_resistor: 0.150
stealthchop_threshold: 0
[stepper_z]
step_pin: PB6
dir_pin: !PB5
enable_pin: !PC3
microsteps: 16
rotation_distance:8
endstop_pin:probe:z_virtual_endstop#PA15 #probe:z_virtual_endstop
position_max: 246
position_min: -5
[tmc2208 stepper_z]
uart_pin: PB14
interpolate: True
run_current: 0.8
stealthchop_threshold: 0
sense_resistor: 0.150
[bltouch]
sensor_pin:PC14
control_pin: PC13
x_offset: 0
y_offset: 27
z_offset: 0
probe_with_touch_mode: true
stow_on_each_sample: false
speed:5
lift_speed:20
[filament_switch_sensor filament_sensor]
switch_pin: !PC15
pause_on_runout: true
[output_pin MainBoardFan]
pin: !PB1
[extruder]
max_extrude_only_distance:1000
max_extrude_cross_section:80
pressure_advance = 0.036
step_pin: PB4
dir_pin: PB3
enable_pin: !PC3
microsteps: 16
rotation_distance: 7.53
nozzle_diameter: 0.400
filament_diameter: 1.750
heater_pin: PA1
sensor_type: EPCOS 100K B57560G104F
sensor_pin: PC5
control = pid
pid_Kp=20.584
pid_Ki=1.737
pid_Kd=60.981
min_temp: 0
max_temp: 320 # Set to 300 for S1 Pro
[heater_bed]
heater_pin: PB2
sensor_type: EPCOS 100K B57560G104F
sensor_pin: PC4
control = pid
pid_kp = 70.652
pid_ki = 1.798
pid_kd = 694.157
min_temp: 0
max_temp: 120 #
temp_offset_flag = True
[verify_heater extruder]
[verify_heater heater_bed]
check_gain_time: 120
heating_gain: 1.0
hysteresis: 10
[temperature_sensor mcu_temp]
sensor_type: temperature_mcu
min_temp: 0
max_temp: 100
[output_pin fan0]
pin:PA0
pwm: True
cycle_time: 0.0100
hardware_pwm: false
value: 0.00
scale: 255
shutdown_value: 0.0
[heater_fan nozzle_fan]
pin: PC1
max_power: 1.0
shutdown_speed: 0
cycle_time: 0.010
hardware_pwm: False
kick_start_time: 0.100
off_below: 0.0
heater: extruder
fan_speed: 1.0
heater_temp: 60.0
[bed_mesh]
speed: 350
mesh_min: 5,10 #need to handle head distance with bl_touch
mesh_max: 215,215 #max probe range
probe_count: 5,5
fade_start: 1
fade_end: 10
fade_target: 0
horizontal_move_z: 8
[input_shaper]
shaper_type_y = mzv
shaper_freq_y = 32.0
shaper_type_x = mzv
shaper_freq_x = 38.5
[mcu rpi]
serial: /tmp/klipper_host_mcu
[adxl345]
cs_pin: rpi:None
spi_speed: 2000000
spi_bus: spidev2.0
axes_map: z,y,x
[resonance_tester]
accel_chip: adxl345
accel_per_hz: 70
probe_points: 117.5,117.5,100
max_freq: 90
-705
View File
@@ -1,705 +0,0 @@
#!/usr/bin/env python3
"""
Ender-3 V3 KE firmware extractor/repacker.
This script handles the stock Creality 7z-wrapped `.img` package, reconstructs
the split payload files, and can rebuild a fresh package from an extracted
rootfs tree.
"""
from __future__ import annotations
import argparse
import hashlib
import os
import shlex
import re
import shutil
import subprocess
import sys
import tempfile
import urllib.request
import zipfile
from pathlib import Path
from typing import List, Sequence, Tuple
import py7zr
PASSWORD = "$1$cxswfile$ZFd0RWFYkJQugbtKVGL9y0"
CHUNK_SIZE = 1024 * 1024
UIMAGE_HEADER_SIZE = 64
DEFAULT_STOCK_ARCHIVE = Path("Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img")
DEFAULT_OVERLAY_DIR = Path("overlay_rootfs")
DEFAULT_VERSION = "11034"
SQUASHFS_TOOLS_URL = "https://infraroot.at/pub/squashfs/windows/squashfs-tools-ng-1.3.2-mingw64.zip"
SQUASHFS_TOOLS_DIR = Path("build") / "_toolcache" / "squashfs-tools-ng-1.3.2-mingw64"
def md5_file(path: Path) -> str:
digest = hashlib.md5()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(4 * 1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def ensure_clean_dir(path: Path) -> None:
if path.exists():
shutil.rmtree(path)
path.mkdir(parents=True, exist_ok=True)
def reconstruct_split_file(src_dir: Path, prefix: str, out_path: Path) -> List[Path]:
pattern = re.compile(rf"^{re.escape(prefix)}\.(\d{{4}})\.[0-9a-f]{{32}}$")
parts: List[Tuple[int, Path]] = []
for entry in src_dir.iterdir():
if not entry.is_file():
continue
match = pattern.match(entry.name)
if match:
parts.append((int(match.group(1)), entry))
if not parts:
raise FileNotFoundError(f"no split files found for {prefix} in {src_dir}")
parts.sort(key=lambda item: item[0])
with out_path.open("wb") as out_handle:
for _, part in parts:
out_handle.write(part.read_bytes())
return [path for _, path in parts]
def write_lines(path: Path, lines: Sequence[str]) -> None:
path.write_text("\n".join(lines) + "\n", encoding="utf-8", newline="\n")
def split_file(src_path: Path, dest_dir: Path, base_name: str) -> Tuple[str, List[Path], List[str]]:
full_md5 = md5_file(src_path)
chunk_md5s: List[str] = []
chunk_paths: List[Path] = []
with src_path.open("rb") as handle:
index = 0
previous_md5 = full_md5
while True:
data = handle.read(CHUNK_SIZE)
if not data:
break
chunk_md5 = hashlib.md5(data).hexdigest()
chunk_name = f"{base_name}.{index:04d}.{previous_md5}"
chunk_path = dest_dir / chunk_name
chunk_path.write_bytes(data)
chunk_paths.append(chunk_path)
chunk_md5s.append(chunk_md5)
previous_md5 = chunk_md5
index += 1
return full_md5, chunk_paths, chunk_md5s
def download_file(url: str, dest: Path) -> None:
dest.parent.mkdir(parents=True, exist_ok=True)
temp_dest = dest.with_name(dest.name + ".download")
if temp_dest.exists():
temp_dest.unlink()
with urllib.request.urlopen(url) as response, temp_dest.open("wb") as out_handle:
shutil.copyfileobj(response, out_handle)
temp_dest.replace(dest)
def resolve_squashfs_tool(tool_name: str) -> Path:
"""
Resolve a squashfs-tools-ng executable for the current host.
On Windows we use the cached MinGW bundle that ships with this repo.
On other platforms we require a native binary from PATH.
"""
if os.name == "nt":
tools_bin = ensure_squashfs_tools(SQUASHFS_TOOLS_DIR, SQUASHFS_TOOLS_URL)
return tools_bin / f"{tool_name}.exe"
resolved = shutil.which(tool_name)
if resolved:
return Path(resolved)
raise FileNotFoundError(
f"{tool_name} not found on PATH. Install squashfs-tools-ng for your Linux distribution "
f"or run the helper on Windows where the bundled MinGW tools are available."
)
def extract_archive(archive: Path, outdir: Path, password: str) -> Path:
ensure_clean_dir(outdir)
with py7zr.SevenZipFile(archive, mode="r", password=password) as zf:
zf.extractall(path=outdir)
top_entries = [entry for entry in outdir.iterdir() if entry.is_dir()]
if len(top_entries) != 1:
raise RuntimeError(f"expected one top-level folder after extraction, found {len(top_entries)}")
return top_entries[0]
def ensure_stock_archive(stock_archive: Path, stock_url: str | None, password: str) -> Path:
if stock_archive.exists():
return stock_archive
if not stock_url:
raise FileNotFoundError(
f"stock archive not found at {stock_archive}; supply --stock-url to download it first"
)
download_file(stock_url, stock_archive)
with py7zr.SevenZipFile(stock_archive, mode="r", password=password) as zf:
zf.test()
return stock_archive
def ensure_squashfs_tools(tools_dir: Path, tools_url: str) -> Path:
bin_dir = tools_dir / "bin"
gensquashfs = bin_dir / "gensquashfs.exe"
rdsquashfs = bin_dir / "rdsquashfs.exe"
if gensquashfs.exists() and rdsquashfs.exists():
return bin_dir
tools_dir.parent.mkdir(parents=True, exist_ok=True)
zip_path = tools_dir.with_suffix(".zip")
if not zip_path.exists():
download_file(tools_url, zip_path)
if tools_dir.exists():
shutil.rmtree(tools_dir)
with zipfile.ZipFile(zip_path) as zf:
zf.extractall(path=tools_dir.parent)
if not gensquashfs.exists() or not rdsquashfs.exists():
raise FileNotFoundError(f"failed to install squashfs tools into {bin_dir}")
return bin_dir
def reconstruct_from_archive_tree(tree_root: Path, workspace: Path) -> dict[str, Path]:
version_file = tree_root / "ota_config.in"
if version_file.exists():
version = read_version(version_file)
ota_dir = tree_root / f"ota_v{version}"
else:
ota_dir = tree_root / f"ota_v{DEFAULT_VERSION}"
if not ota_dir.exists():
matches = sorted([entry for entry in tree_root.iterdir() if entry.is_dir() and entry.name.startswith("ota_v")])
if not matches:
raise FileNotFoundError(f"could not locate ota_v* directory under {tree_root}")
ota_dir = matches[0]
rootfs_full = workspace / "rootfs.squashfs.full"
ximage_full = workspace / "xImage.full"
zero_full = workspace / "zero.bin.full"
rootfs_full.parent.mkdir(parents=True, exist_ok=True)
ximage_full.parent.mkdir(parents=True, exist_ok=True)
zero_full.parent.mkdir(parents=True, exist_ok=True)
reconstruct_split_file(ota_dir, "rootfs.squashfs", rootfs_full)
reconstruct_split_file(ota_dir, "xImage", ximage_full)
reconstruct_split_file(ota_dir, "zero.bin", zero_full)
ximage_payload = workspace / "xImage.payload"
ximage_payload.write_bytes(ximage_full.read_bytes()[UIMAGE_HEADER_SIZE:])
return {
"ota_dir": ota_dir,
"rootfs_full": rootfs_full,
"ximage_full": ximage_full,
"ximage_payload": ximage_payload,
"zero_full": zero_full,
}
def extract_rootfs_to_dir(rootfs_full: Path, outdir: Path) -> None:
ensure_clean_dir(outdir)
rdsquashfs = resolve_squashfs_tool("rdsquashfs")
subprocess.run(
[
str(rdsquashfs),
"--unpack-path",
"/",
"--unpack-root",
str(outdir),
"--chmod",
"--chown",
"--set-times",
"--quiet",
str(rootfs_full),
],
check=True,
)
def describe_rootfs(rootfs_full: Path) -> List[str]:
rdsquashfs = resolve_squashfs_tool("rdsquashfs")
result = subprocess.run(
[str(rdsquashfs), "--describe", str(rootfs_full)],
check=True,
capture_output=True,
text=True,
encoding="utf-8",
errors="replace",
)
return [line.strip() for line in result.stdout.splitlines() if line.strip()]
def describe_rootfs_map(rootfs_full: Path) -> dict[str, str]:
root_entries: dict[str, str] = {}
for line in describe_rootfs(rootfs_full):
parts = shlex.split(line)
if len(parts) < 2:
continue
root_entries[parts[1].lstrip("/")] = line
return root_entries
def packfile_entry(line: str) -> tuple[str, str]:
parts = shlex.split(line)
if len(parts) < 5:
raise ValueError(f"invalid squashfs describe line: {line}")
kind = parts[0]
path = parts[1]
mode = parts[2]
uid = parts[3]
gid = parts[4]
extra = " ".join(parts[5:])
pack_path = path if path.startswith("/") else f"/{path}"
if kind in {"dir", "file"}:
entry = f"{kind} {quote_pack_path(pack_path)} {mode} {uid} {gid}"
elif kind in {"slink", "link"}:
if not extra:
raise ValueError(f"missing link target for {line}")
entry = f"{kind} {quote_pack_path(pack_path)} {mode} {uid} {gid} {extra}"
elif kind == "nod":
if not extra:
raise ValueError(f"missing device metadata for {line}")
entry = f"{kind} {quote_pack_path(pack_path)} {mode} {uid} {gid} {extra}"
else:
raise ValueError(f"unsupported squashfs entry type: {kind}")
return kind, entry
def quote_pack_path(path: str) -> str:
if not any(ch.isspace() for ch in path) and '"' not in path:
return path
return '"' + path.replace("\\", "\\\\").replace('"', '\\"') + '"'
def list_overlay_entries(rootfs_dir: Path, stock_paths: set[str]) -> List[str]:
additions: List[tuple[int, str]] = []
for entry in rootfs_dir.rglob("*"):
relative = entry.relative_to(rootfs_dir).as_posix()
pack_path = f"/{relative}"
if relative in stock_paths:
continue
if entry.is_dir() and not entry.is_symlink():
additions.append((0, f"dir {quote_pack_path(pack_path)} 0755 0 0"))
elif entry.is_symlink():
additions.append((2, f"slink {quote_pack_path(pack_path)} 0777 0 0 {os.readlink(entry)}"))
else:
mode_bits = entry.stat().st_mode & 0o7777
if mode_bits & 0o111 == 0 and relative.startswith("etc/init.d/"):
mode_bits = (mode_bits & ~0o666) | 0o755
mode = oct(mode_bits)[2:]
additions.append((1, f"file {quote_pack_path(pack_path)} {mode} 0 0"))
additions.sort(key=lambda item: (item[0], item[1]))
return [entry for _, entry in additions]
def collect_intentional_overlay_paths(overlay_dir: Path) -> set[str]:
intentional: set[str] = set()
delete_file = overlay_dir / ".delete"
if delete_file.exists():
intentional.update(
str(path).replace("\\", "/")
for path in read_delete_list(overlay_dir)
)
for entry in overlay_dir.rglob("*"):
relative = entry.relative_to(overlay_dir).as_posix()
intentional.add(relative)
return intentional
def should_skip_compare_path(path: str) -> bool:
return any(ord(ch) > 127 for ch in path)
def compare_rootfs_metadata(stock_rootfs_full: Path, rebuilt_rootfs_full: Path, overlay_dir: Path) -> List[str]:
stock_map = describe_rootfs_map(stock_rootfs_full)
rebuilt_map = describe_rootfs_map(rebuilt_rootfs_full)
intentional = collect_intentional_overlay_paths(overlay_dir)
issues: List[str] = []
for path, stock_line in stock_map.items():
if should_skip_compare_path(path):
continue
if path in intentional:
continue
rebuilt_line = rebuilt_map.get(path)
if rebuilt_line is None:
issues.append(f"missing from rebuilt rootfs: {path}")
continue
if rebuilt_line != stock_line:
issues.append(f"metadata mismatch: {path}")
for path, rebuilt_line in rebuilt_map.items():
if should_skip_compare_path(path):
continue
if path in stock_map or path in intentional:
continue
issues.append(f"unexpected extra path in rebuilt rootfs: {path}")
return issues
def write_pack_file(rootfs_dir: Path, stock_rootfs_full: Path, packfile_path: Path) -> None:
lines = describe_rootfs(stock_rootfs_full)
stock_entries: List[str] = []
stock_paths: set[str] = set()
for line in lines:
kind, entry = packfile_entry(line)
path = shlex.split(line)[1]
stock_paths.add(path)
target_path = rootfs_dir / path
if kind in {"file", "dir"}:
if not target_path.exists():
continue
elif kind in {"slink", "link", "nod"}:
if kind in {"slink", "link"} and target_path.exists():
remove_path(target_path)
stock_entries.append(entry)
overlay_entries = list_overlay_entries(rootfs_dir, stock_paths)
packfile_path.write_text("\n".join(stock_entries + overlay_entries) + "\n", encoding="utf-8", newline="\n")
def remove_path(path: Path) -> None:
if not path.exists() and not path.is_symlink():
return
if path.is_dir() and not path.is_symlink():
shutil.rmtree(path)
else:
path.unlink()
def safe_rmtree(path: Path) -> None:
if not path.exists() and not path.is_symlink():
return
if path.is_symlink() or path.is_file():
path.unlink(missing_ok=True)
return
for entry in path.iterdir():
safe_rmtree(entry)
path.rmdir()
def read_delete_list(overlay_dir: Path) -> List[Path]:
delete_file = overlay_dir / ".delete"
if not delete_file.exists():
return []
deletions: List[Path] = []
for raw_line in delete_file.read_text(encoding="utf-8").splitlines():
line = raw_line.strip()
if not line or line.startswith("#"):
continue
deletions.append(Path(line))
return deletions
def apply_overlay_tree(rootfs_dir: Path, overlay_dir: Path) -> None:
if not overlay_dir.exists():
raise FileNotFoundError(f"overlay directory not found: {overlay_dir}")
for relative in read_delete_list(overlay_dir):
target = rootfs_dir / relative
remove_path(target)
for entry in overlay_dir.rglob("*"):
if entry.is_dir():
continue
if entry.name == ".delete" and entry.parent == overlay_dir:
continue
relative = entry.relative_to(overlay_dir)
target = rootfs_dir / relative
target.parent.mkdir(parents=True, exist_ok=True)
remove_path(target)
shutil.copy2(entry, target)
def build_rootfs_from_dir(rootfs_dir: Path, stock_rootfs_full: Path, output_path: Path, workspace: Path) -> None:
gensquashfs = resolve_squashfs_tool("gensquashfs")
packfile_path = workspace / "rootfs.packfile"
write_pack_file(rootfs_dir, stock_rootfs_full, packfile_path)
if output_path.exists():
output_path.unlink()
subprocess.run(
[
str(gensquashfs),
"--pack-file",
str(packfile_path),
"--pack-dir",
str(rootfs_dir),
"--keep-time",
"--block-size",
"131072",
"--compressor",
"gzip",
"--force",
"--quiet",
str(output_path),
],
check=True,
)
def read_version(config_path: Path) -> str:
for line in config_path.read_text(encoding="utf-8").splitlines():
line = line.strip()
if line.startswith("current_version="):
return line.split("=", 1)[1].strip()
raise ValueError(f"current_version not found in {config_path}")
def write_ota_manifest(
staging_root: Path,
version: str,
items: Sequence[Tuple[str, str, Path, Path, List[str]]],
) -> None:
archive_root = staging_root.name
version_dir = staging_root / f"ota_v{version}"
version_dir.mkdir(parents=True, exist_ok=True)
(staging_root / "ota_config.in").write_text(f"current_version={version}\n", encoding="utf-8")
(version_dir / f"ota_v{version}.ok").write_bytes(b"")
ota_update_lines = [f"ota_version={version}", ""]
for img_type, img_name, full_path, md5_path, chunk_md5s in items:
ota_update_lines.extend(
[
f"img_type={img_type}",
f"img_name={img_name}",
f"img_size={full_path.stat().st_size}",
f"img_md5={md5_file(full_path)}",
"",
]
)
sidecar_name = f"ota_md5_{img_name}.{md5_file(full_path)}"
write_lines(version_dir / sidecar_name, chunk_md5s)
write_lines(version_dir / "ota_update.in", ota_update_lines)
def stage_and_pack(
workspace: Path,
version: str,
rootfs_full: Path,
ximage_full: Path,
zero_full: Path,
output_img: Path,
password: str,
) -> Path:
archive_root = f"DarKE_KE_F005_ota_img_V{version}"
staging_root = workspace / "_firmware_stage" / archive_root
ensure_clean_dir(staging_root.parent)
staging_root.mkdir(parents=True, exist_ok=True)
version_dir = staging_root / f"ota_v{version}"
version_dir.mkdir(parents=True, exist_ok=True)
rootfs_md5, rootfs_chunks, rootfs_chunk_md5s = split_file(rootfs_full, version_dir, "rootfs.squashfs")
ximage_md5, ximage_chunks, ximage_chunk_md5s = split_file(ximage_full, version_dir, "xImage")
zero_md5, zero_chunks, zero_chunk_md5s = split_file(zero_full, version_dir, "zero.bin")
(staging_root / "ota_config.in").write_text(f"current_version={version}\n", encoding="utf-8", newline="\n")
(version_dir / f"ota_v{version}.ok").write_bytes(b"")
ota_update_lines = [f"ota_version={version}", ""]
for img_type, img_name, full_path, chunk_md5s, full_md5 in [
("kernel", "xImage", ximage_full, ximage_chunk_md5s, ximage_md5),
("rootfs", "rootfs.squashfs", rootfs_full, rootfs_chunk_md5s, rootfs_md5),
("rtos", "zero.bin", zero_full, zero_chunk_md5s, zero_md5),
]:
ota_update_lines.extend(
[
f"img_type={img_type}",
f"img_name={img_name}",
f"img_size={full_path.stat().st_size}",
f"img_md5={full_md5}",
"",
]
)
sidecar_name = f"ota_md5_{img_name}.{full_md5}"
write_lines(version_dir / sidecar_name, chunk_md5s)
write_lines(version_dir / "ota_update.in", ota_update_lines)
if output_img.exists():
output_img.unlink()
with py7zr.SevenZipFile(output_img, mode="w", password=password) as zf:
zf.writeall(staging_root, arcname=archive_root)
return output_img
def validate_archive(archive: Path, password: str) -> None:
with tempfile.TemporaryDirectory(dir=str(archive.parent)) as tmpdir:
tmp_path = Path(tmpdir)
with py7zr.SevenZipFile(archive, mode="r", password=password) as zf:
zf.extractall(path=tmp_path)
top = next(tmp_path.iterdir(), None)
if top is None or not top.is_dir():
raise RuntimeError("archive validation failed: missing top-level directory")
ota_config = top / "ota_config.in"
if not ota_config.exists():
raise RuntimeError("archive validation failed: missing ota_config.in")
def cmd_extract(args: argparse.Namespace) -> None:
archive = Path(args.archive).expanduser().resolve()
outdir = Path(args.outdir).expanduser().resolve()
workspace = Path(args.workspace).expanduser().resolve()
tree_root = extract_archive(archive, outdir, args.password)
files = reconstruct_from_archive_tree(tree_root, workspace)
extract_rootfs_to_dir(files["rootfs_full"], workspace / "rootfs_extract")
print(f"extracted archive to {outdir}")
print(f"reconstructed rootfs: {files['rootfs_full']}")
print(f"reconstructed kernel: {files['ximage_full']}")
print(f"reconstructed rtos: {files['zero_full']}")
print(f"rootfs unpacked to: {workspace / 'rootfs_extract'}")
def cmd_build(args: argparse.Namespace) -> None:
version = args.version
if not version:
version = DEFAULT_VERSION
build_dir = Path(args.build_dir).expanduser().resolve()
build_dir.mkdir(parents=True, exist_ok=True)
output_img = Path(args.output).expanduser().resolve()
stock_archive = ensure_stock_archive(
Path(args.stock_archive).expanduser().resolve(),
args.stock_url,
args.password,
)
overlay_dir = Path(args.overlay_dir).expanduser().resolve()
workdir = Path(tempfile.mkdtemp(prefix="_firmware_build_", dir=str(build_dir)))
try:
stock_extract_dir = workdir / "stock_extract"
stock_tree = extract_archive(stock_archive, stock_extract_dir, args.password)
stock_files = reconstruct_from_archive_tree(stock_tree, workdir)
rootfs_dir = workdir / "rootfs_work"
extract_rootfs_to_dir(stock_files["rootfs_full"], rootfs_dir)
apply_overlay_tree(rootfs_dir, overlay_dir)
rootfs_full = workdir / "rootfs.squashfs.full"
build_rootfs_from_dir(rootfs_dir, stock_files["rootfs_full"], rootfs_full, workdir)
stage_and_pack(
workspace=workdir,
version=version,
rootfs_full=rootfs_full,
ximage_full=stock_files["ximage_full"],
zero_full=stock_files["zero_full"],
output_img=output_img,
password=args.password,
)
validate_archive(output_img, args.password)
finally:
safe_rmtree(workdir)
print(f"built firmware image: {output_img}")
def cmd_compare(args: argparse.Namespace) -> None:
stock_archive = Path(args.stock_archive).expanduser().resolve()
rebuilt_archive = Path(args.rebuilt_archive).expanduser().resolve()
overlay_dir = Path(args.overlay_dir).expanduser().resolve()
compare_root = Path(args.work_dir).expanduser().resolve()
compare_root.mkdir(parents=True, exist_ok=True)
workdir = Path(tempfile.mkdtemp(prefix="_firmware_compare_", dir=str(compare_root)))
try:
stock_tree = extract_archive(stock_archive, workdir / "stock_extract", args.password)
rebuilt_tree = extract_archive(rebuilt_archive, workdir / "rebuilt_extract", args.password)
stock_files = reconstruct_from_archive_tree(stock_tree, workdir / "stock")
rebuilt_files = reconstruct_from_archive_tree(rebuilt_tree, workdir / "rebuilt")
stock_rootfs_full = stock_files["rootfs_full"]
rebuilt_rootfs_full = rebuilt_files["rootfs_full"]
issues = compare_rootfs_metadata(stock_rootfs_full, rebuilt_rootfs_full, overlay_dir)
if issues:
for issue in issues:
print(issue)
raise SystemExit(1)
finally:
safe_rmtree(workdir)
print("rootfs metadata matches stock outside the overlay")
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description="Extract and rebuild Ender-3 V3 KE firmware images.")
parser.add_argument("--password", default=PASSWORD, help="7z archive password")
subparsers = parser.add_subparsers(dest="command", required=True)
extract = subparsers.add_parser("extract", help="extract an official firmware archive")
extract.add_argument("--archive", required=True, help="path to the stock .img archive")
extract.add_argument("--outdir", default="ke_firmware_extract", help="archive extraction directory")
extract.add_argument("--workspace", default=".", help="workspace for reconstructed artifacts")
extract.set_defaults(func=cmd_extract)
build = subparsers.add_parser("build", help="build a repacked firmware archive")
build.add_argument("--workspace", default=".", help="workspace root")
build.add_argument("--build-dir", default="build", help="directory for intermediate build artifacts")
build.add_argument(
"--stock-archive",
default=str(DEFAULT_STOCK_ARCHIVE),
help="path to the stock OTA image to use as the build base",
)
build.add_argument(
"--stock-url",
default=None,
help="download URL used when the stock OTA image is missing",
)
build.add_argument(
"--overlay-dir",
default=str(DEFAULT_OVERLAY_DIR),
help="directory containing only our custom rootfs changes",
)
build.add_argument(
"--output",
default=str(Path("build") / f"DarKE_KE_F005_ota_img_V{DEFAULT_VERSION}.img"),
help="output .img archive path",
)
build.add_argument("--version", default=DEFAULT_VERSION, help="firmware version to encode in manifests")
build.set_defaults(func=cmd_build)
compare = subparsers.add_parser("compare", help="compare a rebuilt image's rootfs metadata against stock")
compare.add_argument("--stock-archive", required=True, help="path to the stock .img archive")
compare.add_argument("--rebuilt-archive", required=True, help="path to the rebuilt .img archive")
compare.add_argument(
"--overlay-dir",
default=str(DEFAULT_OVERLAY_DIR),
help="overlay tree used to produce the rebuilt image",
)
compare.add_argument("--work-dir", default="build", help="directory for temporary compare artifacts")
compare.set_defaults(func=cmd_compare)
return parser
def main(argv: Sequence[str] | None = None) -> int:
for stream in (sys.stdout, sys.stderr):
try:
stream.reconfigure(encoding="utf-8", errors="replace")
except Exception:
pass
parser = build_parser()
args = parser.parse_args(argv)
args.func(args)
return 0
if __name__ == "__main__":
raise SystemExit(main())
+505
View File
@@ -0,0 +1,505 @@
#!/usr/bin/env python3
"""
OpenKE Firmware Packager for Ender-3 V3 KE and Creality Nebula Pad.
Packages OpenKE kernel (xImage) and rootfs (rootfs.squashfs) along with the board-specific
RTOS firmware (zero.bin) into a stock CrealityOS-supported encrypted 7z OTA (.img) update.
CRITICAL REQUIREMENT (from /etc/ota_bin/local_ota_update.sh):
OTA_UNZIP_FILE_NAME=${OTA_FILE_NAME%.img*}
ota_site=${OTA_FILE_PATH}/${OTA_UNZIP_FILE_NAME}
ota_site_config=$ota_site/ota_config.in
The top-level folder inside the 7z archive MUST EXACTLY MATCH the .img filename (minus .img),
and the version must match Creality's dotted versioning (e.g. 1.1.0.34 > 1.1.0.30).
"""
from __future__ import annotations
import argparse
import hashlib
import os
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
from typing import Dict, List, Sequence, Tuple
# Fixed hardware partition limits for X2000 Creality display boards
KERNEL_PART_MAX_BYTES = 8 * 1024 * 1024 # 8 MiB (mmcblk0p5 / mmcblk0p6)
ROOTFS_PART_MAX_BYTES = 500 * 1024 * 1024 # 500 MiB (mmcblk0p7 / mmcblk0p8)
RTOS_PART_MAX_BYTES = 4 * 1024 * 1024 # 4 MiB (mmcblk0p3 / mmcblk0p4)
CHUNK_SIZE = 1024 * 1024 # 1 MiB
DEFAULT_VERSION = "1.1.0.34"
DEFAULT_OPENKE_ARTIFACTS = Path("..") / "OpenKE" / "artifacts" / "buildroot-halley5-v30-image"
# Target profiles
# NOTE: Filenames must follow Creality's native pattern so master-server and local_ota_update.sh
# recognize them and find the extracted ota_config.in.
TARGET_PROFILES: Dict[str, Dict[str, str]] = {
"ke": {
"board_name": "F005",
"output_pattern": "Ender-3_V3_KE_F005_ota_img_V{version}.img",
"zero_bin": "assets/zero.bin",
"description": "Creality Ender-3 V3 KE (Target F005)",
},
"nebula": {
"board_name": "NEBULA",
"output_pattern": "NEBULA_ota_img_V{version}.img",
"zero_bin": "assets/zero_nebula.bin",
"description": "Creality Nebula Smart Kit / Nebula Pad (Target NEBULA)",
},
}
def derive_creality_password(board_name: str, salt: str = "cxswfile") -> str:
"""
Derives Creality's standard MD5-crypt password for a given board:
mkpasswd -m md5 "${BOARD_NAME}C3_7e_bz" -S cxswfile
Implemented in pure Python to eliminate platform dependencies.
"""
key = f"{board_name}C3_7e_bz"
magic = "$1$"
pw = key.encode("utf-8")
s = salt.encode("utf-8")
ctx = hashlib.md5(pw + magic.encode("utf-8") + s)
alt = hashlib.md5(pw + s + pw).digest()
for i in range(len(pw), 0, -16):
ctx.update(alt[: min(i, 16)])
i = len(pw)
while i > 0:
if i & 1:
ctx.update(b"\x00")
else:
ctx.update(pw[:1])
i >>= 1
digest = ctx.digest()
for idx in range(1000):
c = hashlib.md5()
if idx & 1:
c.update(pw)
else:
c.update(digest)
if idx % 3:
c.update(s)
if idx % 7:
c.update(pw)
if idx & 1:
c.update(digest)
else:
c.update(pw)
digest = c.digest()
b64 = "./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
def to64(v: int, n: int) -> str:
ret = []
for _ in range(n):
ret.append(b64[v & 0x3F])
v >>= 6
return "".join(ret)
res = magic + salt + "$"
d = digest
res += to64((d[0] << 16) | (d[6] << 8) | d[12], 4)
res += to64((d[1] << 16) | (d[7] << 8) | d[13], 4)
res += to64((d[2] << 16) | (d[8] << 8) | d[14], 4)
res += to64((d[3] << 16) | (d[9] << 8) | d[15], 4)
res += to64((d[4] << 16) | (d[10] << 8) | d[5], 4)
res += to64(d[11], 2)
return res
def compute_md5(path: Path) -> str:
"""Compute MD5 digest of a file in 4MB streaming chunks."""
digest = hashlib.md5()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(4 * 1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def compute_sha256(path: Path) -> str:
"""Compute SHA256 digest of a file in 4MB streaming chunks."""
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(4 * 1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def split_into_chunks(src_path: Path, dest_dir: Path, base_name: str) -> Tuple[str, List[str]]:
"""
Split payload into 1 MiB chunks per Creality OTA specification:
<base_name>.<index:04d>.<prev_md5>
where index 0000 uses the full file MD5 as prev_md5.
Returns (full_file_md5, list_of_chunk_md5s).
"""
full_md5 = compute_md5(src_path)
chunk_md5s: List[str] = []
with src_path.open("rb") as handle:
index = 0
prev_md5 = full_md5
while True:
data = handle.read(CHUNK_SIZE)
if not data:
break
chunk_md5 = hashlib.md5(data).hexdigest()
chunk_name = f"{base_name}.{index:04d}.{prev_md5}"
chunk_path = dest_dir / chunk_name
chunk_path.write_bytes(data)
chunk_md5s.append(chunk_md5)
prev_md5 = chunk_md5
index += 1
return full_md5, chunk_md5s
def verify_manifest(manifest_path: Path, ximage_path: Path, rootfs_path: Path) -> None:
"""Verify xImage and rootfs.squashfs match build-manifest.txt if present."""
if not manifest_path.exists():
print(f"[*] Manifest {manifest_path} not found, skipping manifest verification.")
return
print(f"[*] Verifying inputs against build manifest: {manifest_path}")
manifest_data = {}
for line in manifest_path.read_text(encoding="utf-8").splitlines():
line = line.strip()
if "=" in line and not line.startswith("#"):
key, val = line.split("=", 1)
manifest_data[key.strip()] = val.strip()
expected_ximage_size = manifest_data.get("xImage_size")
expected_ximage_sha = manifest_data.get("xImage_sha256")
expected_rootfs_size = manifest_data.get("rootfs_squashfs_size")
expected_rootfs_sha = manifest_data.get("rootfs_squashfs_sha256")
if expected_ximage_size and ximage_path.stat().st_size != int(expected_ximage_size):
raise ValueError(
f"xImage size mismatch: {ximage_path.stat().st_size} bytes vs expected {expected_ximage_size}"
)
if expected_ximage_sha:
actual_ximage_sha = compute_sha256(ximage_path)
if actual_ximage_sha != expected_ximage_sha:
raise ValueError(f"xImage SHA256 mismatch: {actual_ximage_sha} != {expected_ximage_sha}")
if expected_rootfs_size and rootfs_path.stat().st_size != int(expected_rootfs_size):
raise ValueError(
f"rootfs.squashfs size mismatch: {rootfs_path.stat().st_size} bytes vs expected {expected_rootfs_size}"
)
if expected_rootfs_sha:
actual_rootfs_sha = compute_sha256(rootfs_path)
if actual_rootfs_sha != expected_rootfs_sha:
raise ValueError(
f"rootfs.squashfs SHA256 mismatch: {actual_rootfs_sha} != {expected_rootfs_sha}"
)
print(" [+] Manifest verification passed: exact size & SHA256 match.")
def pack_archive(staging_root: Path, archive_name: str, output_img: Path, password: str) -> None:
"""Create encrypted 7z archive using 7z CLI (if available) or py7zr."""
output_img.parent.mkdir(parents=True, exist_ok=True)
if output_img.exists():
output_img.unlink()
has_7z = shutil.which("7z") is not None
if has_7z:
print("[*] Compressing encrypted 7z archive using system 7z CLI (multi-threaded)...")
cmd = [
"7z", "a",
"-t7z",
f"-p{password}",
"-mhe=on",
"-mx=4",
str(output_img),
archive_name,
]
res = subprocess.run(cmd, cwd=str(staging_root.parent), stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
if res.returncode != 0:
raise RuntimeError(f"7z command failed with code {res.returncode}:\n{res.stdout}")
else:
print("[*] System 7z not found. Compressing using Python py7zr (single-threaded)...")
try:
import py7zr
except ImportError:
raise RuntimeError("Neither 7z CLI nor python 'py7zr' is available. Install 7zip or pip install py7zr.")
with py7zr.SevenZipFile(output_img, mode="w", password=password, header_encryption=True) as zf:
zf.writeall(staging_root, arcname=archive_name)
def verify_archive(archive_path: Path, password: str) -> None:
"""Verify archive integrity and password decryption."""
print(f"[*] Verifying archive integrity: {archive_path}")
has_7z = shutil.which("7z") is not None
if has_7z:
cmd = ["7z", "t", f"-p{password}", str(archive_path)]
res = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
if res.returncode != 0:
raise RuntimeError(f"Archive verification failed:\n{res.stderr or res.stdout}")
else:
import py7zr
with py7zr.SevenZipFile(archive_path, mode="r", password=password) as zf:
if not zf.test():
raise RuntimeError("Archive validation failed via py7zr.")
print(" [+] Archive verification passed: 100% valid encrypted 7z envelope.")
def build_openke_ota_image(
ximage_path: Path,
rootfs_path: Path,
zero_path: Path,
manifest_path: Path | None,
output_img: Path,
version: str,
password: str,
target_desc: str = "Creality Display",
) -> Path:
"""End-to-end packaging function for OpenKE OTA .img."""
print("=" * 65)
print(f" OpenKE Firmware Packager: {target_desc}")
print("=" * 65)
if not ximage_path.exists():
raise FileNotFoundError(f"xImage not found: {ximage_path}")
if not rootfs_path.exists():
raise FileNotFoundError(f"rootfs.squashfs not found: {rootfs_path}")
if not zero_path.exists():
raise FileNotFoundError(f"zero.bin (RTOS firmware) not found at {zero_path}.")
ximage_size = ximage_path.stat().st_size
rootfs_size = rootfs_path.stat().st_size
zero_size = zero_path.stat().st_size
print("[*] Partition Budget Preflight:")
ximage_pct = (ximage_size / KERNEL_PART_MAX_BYTES) * 100
ximage_free = (KERNEL_PART_MAX_BYTES - ximage_size) / (1024 * 1024)
print(f" - Kernel: {ximage_size:,} B / {KERNEL_PART_MAX_BYTES:,} B ({ximage_pct:.1f}% used, {ximage_free:.2f} MiB free)")
if ximage_size > KERNEL_PART_MAX_BYTES:
raise ValueError(f"xImage ({ximage_size} B) exceeds max partition size ({KERNEL_PART_MAX_BYTES} B)!")
rootfs_pct = (rootfs_size / ROOTFS_PART_MAX_BYTES) * 100
rootfs_free = (ROOTFS_PART_MAX_BYTES - rootfs_size) / (1024 * 1024)
print(f" - Rootfs: {rootfs_size:,} B / {ROOTFS_PART_MAX_BYTES:,} B ({rootfs_pct:.1f}% used, {rootfs_free:.2f} MiB free)")
if rootfs_size > ROOTFS_PART_MAX_BYTES:
raise ValueError(f"rootfs.squashfs ({rootfs_size} B) exceeds max partition size ({ROOTFS_PART_MAX_BYTES} B)!")
print(f" - RTOS: {zero_size:,} B / {RTOS_PART_MAX_BYTES:,} B")
if zero_size > RTOS_PART_MAX_BYTES:
raise ValueError(f"zero.bin ({zero_size} B) exceeds max partition size ({RTOS_PART_MAX_BYTES} B)!")
if manifest_path:
verify_manifest(manifest_path, ximage_path, rootfs_path)
# CRITICAL: Creality's local_ota_update.sh expects:
# OTA_UNZIP_FILE_NAME=${OTA_FILE_NAME%.img*}
# ota_site=${OTA_FILE_PATH}/${OTA_UNZIP_FILE_NAME}
# ota_site_config=$ota_site/ota_config.in
# Therefore the root directory inside the archive MUST exactly equal output_img.stem!
archive_name = output_img.name
if archive_name.endswith(".img"):
archive_name = archive_name[:-4]
with tempfile.TemporaryDirectory(prefix="openke_ota_stage_") as tmpdir:
stage_dir = Path(tmpdir)
staging_root = stage_dir / archive_name
version_dir = staging_root / f"ota_v{version}"
version_dir.mkdir(parents=True, exist_ok=True)
print(f"[*] Packaging archive layout: {archive_name}/ota_v{version}/")
print("[*] Slicing and chunking payloads (1 MiB slices):")
print(" - Slicing xImage...")
ximage_md5, ximage_chunk_md5s = split_into_chunks(ximage_path, version_dir, "xImage")
print(f" Total chunks: {len(ximage_chunk_md5s)}, Full MD5: {ximage_md5}")
print(" - Slicing rootfs.squashfs...")
rootfs_md5, rootfs_chunk_md5s = split_into_chunks(rootfs_path, version_dir, "rootfs.squashfs")
print(f" Total chunks: {len(rootfs_chunk_md5s)}, Full MD5: {rootfs_md5}")
print(" - Slicing zero.bin...")
zero_md5, zero_chunk_md5s = split_into_chunks(zero_path, version_dir, "zero.bin")
print(f" Total chunks: {len(zero_chunk_md5s)}, Full MD5: {zero_md5}")
print("[*] Generating Creality OTA manifests and MD5 sidecars...")
(staging_root / "ota_config.in").write_text(f"current_version={version}\n", encoding="utf-8", newline="\n")
(version_dir / f"ota_v{version}.ok").write_bytes(b"\n")
ota_update_lines = [
f"ota_version={version}",
"",
"img_type=kernel",
"img_name=xImage",
f"img_size={ximage_size}",
f"img_md5={ximage_md5}",
"",
"img_type=rootfs",
"img_name=rootfs.squashfs",
f"img_size={rootfs_size}",
f"img_md5={rootfs_md5}",
"",
"img_type=rtos",
"img_name=zero.bin",
f"img_size={zero_size}",
f"img_md5={zero_md5}",
"",
]
(version_dir / "ota_update.in").write_text("\n".join(ota_update_lines) + "\n", encoding="utf-8", newline="\n")
(version_dir / f"ota_md5_xImage.{ximage_md5}").write_text("\n".join(ximage_chunk_md5s) + "\n", encoding="utf-8", newline="\n")
(version_dir / f"ota_md5_rootfs.squashfs.{rootfs_md5}").write_text("\n".join(rootfs_chunk_md5s) + "\n", encoding="utf-8", newline="\n")
(version_dir / f"ota_md5_zero.bin.{zero_md5}").write_text("\n".join(zero_chunk_md5s) + "\n", encoding="utf-8", newline="\n")
pack_archive(staging_root, archive_name, output_img, password)
verify_archive(output_img, password)
print("=" * 65)
print(" SUCCESS: OpenKE OTA Image Created Successfully!")
print(f" Target Image: {output_img}")
print(f" Size: {output_img.stat().st_size:,} bytes ({output_img.stat().st_size / (1024*1024):.2f} MiB)")
print(f" Package Ver: {version}")
print("=" * 65)
print(" FLASHING INSTRUCTIONS:")
print(" 1. Format a USB flash drive as FAT32.")
print(f" 2. Copy '{output_img.name}' to the root of the USB drive.")
print(" 3. Insert the USB drive into the printer / Nebula Pad front USB port.")
print(" 4. On the touchscreen: Settings -> Firmware Update -> Local Update.")
print(" 5. Select the update and confirm. The device will flash OpenKE to")
print(" Slot 2, point the bootloader to ota:kernel2, and automatically reboot.")
print(" 6. First boot will seed /usr/data/openke and launch Klipper & Moonraker.")
print("=" * 65)
return output_img
def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Package OpenKE build artifacts into stock CrealityOS-compatible USB update (.img) packages."
)
parser.add_argument(
"--target",
choices=["ke", "nebula", "all"],
default="ke",
help="Target hardware profile: 'ke' (Ender-3 V3 KE), 'nebula' (Nebula Pad), or 'all' (build both) (default: ke)",
)
parser.add_argument(
"--artifacts-dir",
type=Path,
default=None,
help="Path to OpenKE build artifacts directory (contains xImage, rootfs.squashfs, and build-manifest.txt)",
)
parser.add_argument("--ximage", type=Path, default=None, help="Explicit path to OpenKE xImage")
parser.add_argument("--rootfs", type=Path, default=None, help="Explicit path to OpenKE rootfs.squashfs")
parser.add_argument("--manifest", type=Path, default=None, help="Explicit path to build-manifest.txt")
parser.add_argument(
"--zero-bin",
type=Path,
default=None,
help="Path to RTOS firmware (zero.bin). Defaults to profile asset.",
)
parser.add_argument(
"--version",
default=DEFAULT_VERSION,
help=f"Firmware package version recognized by stock updater (default: {DEFAULT_VERSION})",
)
parser.add_argument(
"--output",
type=Path,
default=None,
help="Output image path (applicable when packaging a single target)",
)
parser.add_argument(
"--password",
default=None,
help="Custom Creality 7z encryption password (defaults to derived per-board key)",
)
return parser.parse_args(argv)
def main(argv: Sequence[str] | None = None) -> int:
args = parse_args(argv)
artifacts_dir = args.artifacts_dir
if artifacts_dir is None and (args.ximage is None or args.rootfs is None):
if DEFAULT_OPENKE_ARTIFACTS.exists():
artifacts_dir = DEFAULT_OPENKE_ARTIFACTS
ximage = args.ximage
rootfs = args.rootfs
manifest = args.manifest
if artifacts_dir:
artifacts_dir = artifacts_dir.resolve()
if ximage is None:
ximage = artifacts_dir / "xImage"
if rootfs is None:
rootfs = artifacts_dir / "rootfs.squashfs"
if manifest is None and (artifacts_dir / "build-manifest.txt").exists():
manifest = artifacts_dir / "build-manifest.txt"
if ximage is None or rootfs is None:
print("Error: Must provide --artifacts-dir OR both --ximage and --rootfs", file=sys.stderr)
return 1
ximage = ximage.resolve()
rootfs = rootfs.resolve()
if manifest:
manifest = manifest.resolve()
version = args.version
targets_to_build = ["ke", "nebula"] if args.target == "all" else [args.target]
if args.output and len(targets_to_build) > 1:
print("Error: --output cannot be used with --target all (use default paths).", file=sys.stderr)
return 1
for tgt in targets_to_build:
profile = TARGET_PROFILES[tgt]
board_name = profile["board_name"]
desc = profile["description"]
password = args.password
if password is None:
password = derive_creality_password(board_name)
zero_bin = args.zero_bin
if zero_bin is None:
script_dir = Path(__file__).parent.parent
zero_bin = script_dir / profile["zero_bin"]
zero_bin = zero_bin.resolve()
output = args.output
if output is None:
out_filename = profile["output_pattern"].format(version=version)
output = Path("build") / out_filename
output = output.resolve()
try:
build_openke_ota_image(
ximage_path=ximage,
rootfs_path=rootfs,
zero_path=zero_bin,
manifest_path=manifest,
output_img=output,
version=version,
password=password,
target_desc=desc,
)
except Exception as exc:
print(f"\n[!] ERROR ({tgt}): {exc}", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
sys.exit(main())