Compare commits
2
Commits
c0dd058ae7
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
666a0726d3 | ||
|
|
6a43841867 |
+12
-3
@@ -1,4 +1,13 @@
|
|||||||
# Generated firmware artifacts
|
# Build output
|
||||||
/build/
|
/build/
|
||||||
tools/__pycache__/
|
|
||||||
**/__pycache__/
|
# Firmware images (allow tracked official stock baselines)
|
||||||
|
*.img
|
||||||
|
!Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img
|
||||||
|
!NEBULA_ota_img_V1.1.0.30.img
|
||||||
|
|
||||||
|
# Python cache & temporary files
|
||||||
|
__pycache__/
|
||||||
|
*.py[cod]
|
||||||
|
*$py.class
|
||||||
|
*.tmp
|
||||||
|
|||||||
+36
-41
@@ -1,50 +1,45 @@
|
|||||||
# Firmware Changelog
|
# Firmware Changelog
|
||||||
|
|
||||||
Differences between the stock Creality Ender-3 V3 KE image and our custom build.
|
Differences, release milestones, and architecture changes across this workspace.
|
||||||
|
|
||||||
## Baseline
|
---
|
||||||
|
|
||||||
- Stock image: `Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img`
|
## 1.0.0 — OpenKE Stock USB Installer (2026-09-24)
|
||||||
- Custom build line: `DarKE_KE_F005_ota_img_V11034.img`
|
|
||||||
|
|
||||||
## Firmware Changes
|
### New Features & Architecture Transformation
|
||||||
|
* **Transformed Repo into OpenKE Installer**: Repurposed workspace from a stock pre-root patcher into a full-featured stock-compatible installer generator for **OpenKE** (Linux 6.6-rt + Klipper/Moonraker/GuppyScreen).
|
||||||
|
* **Zero-Disassembly / Single-USB Install**: Users can install OpenKE directly from an unmodified stock printer or Nebula Pad via the touchscreen UI without needing Ingenic mask-ROM recovery mode, cloner tools, or prior root access.
|
||||||
|
* **Dual Target Hardware Profiles**:
|
||||||
|
* **Ender-3 V3 KE** (Board target `F005`)
|
||||||
|
* **Creality Nebula Smart Kit / Nebula Pad** (Board target `NEBULA`)
|
||||||
|
* **Standalone Build Script (`build_openke_image.sh`)**:
|
||||||
|
* Automates preflight checks, asset selection, packaging, and validation.
|
||||||
|
* Supports `--target {ke,nebula,all}`, `--version`, `--artifacts`, and `--output`.
|
||||||
|
* **Direct OpenKE Packager (`tools/pack_openke.py`)**:
|
||||||
|
* Directly encapsulates production OpenKE `xImage` and `rootfs.squashfs` without requiring SquashFS re-compression.
|
||||||
|
* Validates hardware partition budgets: Kernel $\le$ 8 MiB (5.54 MiB used), Rootfs $\le$ 500 MiB (330.85 MiB used).
|
||||||
|
* Validates SHA256 checksums against `build-manifest.txt`.
|
||||||
|
* Generates 1 MiB payload chunks, MD5 hash sidecars, and Creality manifests.
|
||||||
|
* Implements pure-Python Unix MD5-crypt password derivation for any Creality board.
|
||||||
|
* Automatically verifies created 7z archives upon completion.
|
||||||
|
* **Cached RTOS Binaries**:
|
||||||
|
* Extracted and cached `assets/zero.bin` (KE, 432 KB) and `assets/zero_nebula.bin` (Nebula, 424 KB), making builds completely standalone without needing the 118 MB stock firmware images.
|
||||||
|
|
||||||
- Added `overlay_rootfs/etc/init.d/S05agree_root` to create `/usr/data/creality/userdata/user_agree_root` at boot.
|
### Forensics, Bug Fixes & Live Qualification
|
||||||
- Updated the version metadata used by the UI:
|
* **Resolved OTA Extraction Failure on Stock Updater**:
|
||||||
- `overlay_rootfs/etc/ota_info`
|
* *Root Cause*: Creality's `/etc/ota_bin/local_ota_update.sh` computes `OTA_UNZIP_FILE_NAME=${OTA_FILE_NAME%.img*}` and expects `ota_config.in` at `<TEMP_DIR>/<OTA_UNZIP_FILE_NAME>/ota_config.in`. When the archive file and internal folder names drifted, extraction aborted.
|
||||||
- `overlay_rootfs/usr/share/klipper/config/F005/printer.cfg`
|
* *Fix*: Ensured root directory inside the 7z archive strictly matches the `.img` filename without `.img`.
|
||||||
- Current display version: `v1.1.0.33`
|
* **Resolved Version Comparison Rejection**:
|
||||||
|
* *Root Cause*: Creality's `master-server` (`UpgradeManager.c`) parses versions after `ota_img_V` using regex `[0-9]{1,3}`. Raw integers like `11034` failed comparison against dotted stock versions.
|
||||||
|
* *Fix*: Formatted all release versions with standard dotted numbers (e.g. `1.1.0.34`), ensuring `master-server` and `local_ota_update.sh` recognize the update as newer than stock (`1.1.0.30` / `1.1.0.17`).
|
||||||
|
* **Live Qualification**:
|
||||||
|
* Successfully verified on real Creality Nebula Pad hardware via USB flash drive.
|
||||||
|
|
||||||
## Package Identity
|
---
|
||||||
|
|
||||||
- OTA package version: `11034`
|
## 0.0.1 — Pre-Rooted Stock Firmware (Legacy)
|
||||||
- UI/display version: `v1.1.0.33`
|
|
||||||
- The package still flashes through the stock USB updater flow.
|
|
||||||
|
|
||||||
## Build System Changes
|
* Baseline: `Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img`
|
||||||
|
* Added `overlay_rootfs/etc/init.d/S05agree_root` to create `/usr/data/creality/userdata/user_agree_root` at boot.
|
||||||
- `tools/ke_firmware.py` now:
|
* Enabled root access on stock CrealityOS.
|
||||||
- extracts the stock OTA archive
|
* Provided `tools/ke_firmware.py` for SquashFS metadata-preserving rebuilds.
|
||||||
- unpacks the stock SquashFS rootfs
|
|
||||||
- applies the overlay
|
|
||||||
- rebuilds the rootfs
|
|
||||||
- repacks the archive
|
|
||||||
- Repacking preserves stock metadata:
|
|
||||||
- symlinks stay symlinks
|
|
||||||
- executable bits stay intact
|
|
||||||
- ownership and timestamps are preserved where applicable
|
|
||||||
- Added archive validation and stock-vs-rebuilt compare support.
|
|
||||||
- New `etc/init.d/*` overlay scripts are forced to pack as executable.
|
|
||||||
|
|
||||||
## Verified
|
|
||||||
|
|
||||||
- The rebuilt image matches stock rootfs metadata outside the overlay.
|
|
||||||
- The custom image rebuilds cleanly and flashes successfully.
|
|
||||||
- The root-agree file is created at boot by init, not baked in as a static overlay file.
|
|
||||||
|
|
||||||
|
|
||||||
# Changelog:
|
|
||||||
|
|
||||||
|
|
||||||
## 0.0.1
|
|
||||||
Pre-Rooted
|
|
||||||
|
|||||||
+51
-59
@@ -1,69 +1,61 @@
|
|||||||
# Ender-3 V3 KE Firmware Notes
|
# Firmware Architecture & Packaging Notes
|
||||||
|
|
||||||
## Stock firmware source
|
This document contains technical reference information on stock firmware layouts, encryption passwords, partition budgets, and packaging mechanics for the **Ender-3 V3 KE** and **Creality Nebula Pad**.
|
||||||
|
|
||||||
- Downloaded official image: `Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img`
|
---
|
||||||
- Official version: `V1.1.0.17`
|
|
||||||
- Manifest layers inside the archive:
|
|
||||||
- `xImage` (`kernel`)
|
|
||||||
- `rootfs.squashfs` (`rootfs`)
|
|
||||||
- `zero.bin` (`rtos`)
|
|
||||||
|
|
||||||
## 7z archive password
|
## 1. Stock Firmware Baselines
|
||||||
|
|
||||||
The official Creality firmware archive is password-protected.
|
| Target | Official Baseline Image | Version | Board Name | Derived 7z Encryption Password |
|
||||||
|
| :--- | :--- | :--- | :--- | :--- |
|
||||||
|
| **Ender-3 V3 KE** | `Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img` | `V1.1.0.17` | `F005` | `$1$cxswfile$ZFd0RWFYkJQugbtKVGL9y0` |
|
||||||
|
| **Nebula Pad** | `NEBULA_ota_img_V1.1.0.30.img` | `V1.1.0.30` | `NEBULA` | `$1$cxswfile$XG7ANbYX3bq2H2xmmwrka.` |
|
||||||
|
|
||||||
Password:
|
### Password Derivation Algorithm
|
||||||
|
```bash
|
||||||
`$1$cxswfile$ZFd0RWFYkJQugbtKVGL9y0`
|
openssl passwd -1 -salt cxswfile "${BOARD_NAME}C3_7e_bz"
|
||||||
|
```
|
||||||
## Workspace layout
|
Or in Python:
|
||||||
|
```python
|
||||||
- Stock archive cache: `./Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img`
|
derive_creality_password(board_name) # implemented in tools/pack_openke.py
|
||||||
- Overlay tree for our changes: `./overlay_rootfs/`
|
|
||||||
- Build cache: `./build/_toolcache/squashfs-tools-ng-1.3.2-mingw64/`
|
|
||||||
- Temporary build workspace: whichever directory you pass with `--build-dir`
|
|
||||||
|
|
||||||
## Useful edit points in the stock rootfs
|
|
||||||
|
|
||||||
- Klipper startup/service hook: `rootfs_extract/etc/init.d/S55klipper_service`
|
|
||||||
- WebRTC startup/service hook: `rootfs_extract/etc/init.d/S97webrtc`
|
|
||||||
- Stock printer profile:
|
|
||||||
- `rootfs_extract/usr/share/klipper/config/F005/printer.cfg`
|
|
||||||
- `rootfs_extract/usr/share/klipper/config/F005/gcode_macro.cfg`
|
|
||||||
- `rootfs_extract/usr/share/klipper/config/F005/printer_params.cfg`
|
|
||||||
- Runtime printer config is copied into `/usr/data/printer_data/config/` on the device
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- The shipped `.img` file is a 7z archive, not a raw flash image.
|
|
||||||
- The build flow starts from the stock archive, overlays `overlay_rootfs/`, and repacks the result.
|
|
||||||
- The helper uses native `rdsquashfs` and `gensquashfs` on Linux. Install
|
|
||||||
`squashfs-tools-ng` so those binaries are on `PATH`. On Windows it uses the
|
|
||||||
bundled `rdsquashfs.exe` / `gensquashfs.exe` tools from `squashfs-tools-ng`.
|
|
||||||
- The repack path is metadata-preserving: it reuses the stock SquashFS layout so symlinks and execute bits survive the round trip.
|
|
||||||
- `xImage.full` is a stock U-Boot `uImage` kernel blob. Its raw payload can be derived if kernel-side edits are needed.
|
|
||||||
- Stock-updater-facing version scheme in this workspace: `11034`
|
|
||||||
- UI/display version stays on the dotted printer-side label `v1.1.0.33`
|
|
||||||
- The display/version string is sourced from `overlay_rootfs/etc/ota_info`.
|
|
||||||
- The firmware image is intended to flash through the stock USB updater without any manual printer-side file edits.
|
|
||||||
- The helper build process keeps the stock updater happy by using the numeric package version and stock archive layout.
|
|
||||||
|
|
||||||
## Rebuild
|
|
||||||
|
|
||||||
Use the helper script:
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
python .\tools\ke_firmware.py build --build-dir D:\DarKE_build --stock-archive .\Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img --overlay-dir .\overlay_rootfs --output .\build\DarKE_KE_F005_ota_img_V11034.img
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Compare the rebuilt rootfs against stock, allowing only the overlay tree to differ:
|
---
|
||||||
|
|
||||||
```powershell
|
## 2. Partition Capacities & Budget Qualification
|
||||||
python .\tools\ke_firmware.py compare --stock-archive .\Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img --rebuilt-archive .\build\DarKE_KE_F005_ota_img_V11034.img --overlay-dir .\overlay_rootfs
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stock-safe flash summary
|
Creality's X2000 partition table enforces hard upper bounds on flashable images:
|
||||||
|
|
||||||
- Put the generated `DarKE_KE_F005_ota_img_V11034.img` on a USB drive.
|
| Sub-Payload | Target Partition | Hardware Limit | OpenKE Payload Size | Headroom |
|
||||||
- Flash it with the printer's built-in updater.
|
| :--- | :--- | :--- | :--- | :--- |
|
||||||
|
| **`xImage`** (Linux kernel) | `kernel` (`p5`) / `kernel2` (`p6`) | **8 MiB** (8,388,608 B) | **5.54 MiB** (5,541,952 B) | +2.71 MiB (66.1% used) |
|
||||||
|
| **`rootfs.squashfs`** | `rootfs` (`p7`) / `rootfs2` (`p8`) | **500 MiB** (524,288,000 B) | **330.85 MiB** (346,923,008 B) | +169.15 MiB (66.2% used) |
|
||||||
|
| **`zero.bin`** (RTOS) | `rtos` (`p3`) / `rtos2` (`p4`) | **4 MiB** (4,194,304 B) | **0.41–0.42 MiB** (~430 KiB) | +3.58 MiB (10.3% used) |
|
||||||
|
|
||||||
|
Both kernel and rootfs fit comfortably with substantial safety margins.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Tooling Overview
|
||||||
|
|
||||||
|
* **[`build_openke_image.sh`](file:///root/Development/OpenKE/DarKE/build_openke_image.sh)**:
|
||||||
|
Top-level script for building ready-to-flash OpenKE `.img` files directly from build artifacts.
|
||||||
|
* **[`tools/pack_openke.py`](file:///root/Development/OpenKE/DarKE/tools/pack_openke.py)**:
|
||||||
|
Core Python packager. Validates partition budgets, verifies SHA256 against `build-manifest.txt`, slices payloads into 1 MiB chunks with MD5 sidecars, derives board passwords, and compresses encrypted 7z envelopes.
|
||||||
|
* **[`assets/zero.bin`](file:///root/Development/OpenKE/DarKE/assets/zero.bin)**:
|
||||||
|
Cached RTOS binary for the Ender-3 V3 KE (`F005`).
|
||||||
|
* **[`assets/zero_nebula.bin`](file:///root/Development/OpenKE/DarKE/assets/zero_nebula.bin)**:
|
||||||
|
Cached RTOS binary for the Nebula Smart Kit / Nebula Pad (`NEBULA`).
|
||||||
|
* **[`tools/ke_firmware.py`](file:///root/Development/OpenKE/DarKE/tools/ke_firmware.py)**:
|
||||||
|
Legacy utility for extracting stock images, overlaying files onto stock rootfs, and repacking pre-rooted stock images.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Key Rules for Creality OTA Packages
|
||||||
|
|
||||||
|
From reverse-engineering `/etc/ota_bin/local_ota_update.sh`:
|
||||||
|
|
||||||
|
1. **Folder Name Parity**: The root directory inside the 7z archive **must match the `.img` filename** without the `.img` extension.
|
||||||
|
- Example: `NEBULA_ota_img_V1.1.0.34.img` $\rightarrow$ folder inside 7z must be `NEBULA_ota_img_V1.1.0.34/`.
|
||||||
|
2. **Dotted Versioning**: The version must be formatted with dots (e.g. `1.1.0.34`), not a raw integer, so `master-server`'s `[0-9]{1,3}` parser can compare versions.
|
||||||
|
3. **Chunking**: Chunks are 1,048,576 bytes each. Chunk `0000` points to the whole-file MD5, and subsequent chunks chain the previous chunk's MD5.
|
||||||
|
|||||||
Binary file not shown.
@@ -1,49 +1,146 @@
|
|||||||
# Ender-3 V3 KE firmware workspace
|
# OpenKE Stock USB Installer & Firmware Workspace
|
||||||
|
|
||||||
This repo contains a stock KE firmware image, a small overlay tree of our changes, and a helper script for rebuilding an `.img` package.
|
A standalone utility to package **[OpenKE](https://github.com/coreflake1/OpenKE)** (Linux 6.6-rt + Klipper/Moonraker/GuppyScreen) directly into stock CrealityOS-supported OTA update (`.img`) containers.
|
||||||
|
|
||||||
## Files
|
This enables a **zero-disassembly, single-USB installation of OpenKE straight from an unmodified stock printer or Nebula Pad touchscreen** without requiring Ingenic mask-ROM recovery mode, cloner utilities, or prior root access.
|
||||||
|
|
||||||
- `FIRMWARE_NOTES.md`: archive password and firmware layout notes
|
---
|
||||||
- `tools/ke_firmware.py`: extract/rebuild helper
|
|
||||||
- `requirements.txt`: Python dependency list for the helper script
|
|
||||||
|
|
||||||
## Python setup
|
## Supported Hardware
|
||||||
|
|
||||||
Install the helper dependency with:
|
| Device | Board Identifier | Official Stock Baseline | Generated OTA Installer Package |
|
||||||
|
| :--- | :--- | :--- | :--- |
|
||||||
|
| **Creality Ender-3 V3 KE** | `F005` | `Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img` | [`build/Ender-3_V3_KE_F005_ota_img_V1.1.0.34.img`](build/Ender-3_V3_KE_F005_ota_img_V1.1.0.34.img) |
|
||||||
|
| **Creality Nebula Pad** | `NEBULA` | `NEBULA_ota_img_V1.1.0.30.img` | [`build/NEBULA_ota_img_V1.1.0.34.img`](build/NEBULA_ota_img_V1.1.0.34.img) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Table of Contents
|
||||||
|
|
||||||
|
1. [Quick Start](#1-quick-start)
|
||||||
|
2. [Flashing Guide](#2-flashing-guide)
|
||||||
|
3. [Safety & Dual-Slot A/B Architecture](#3-safety--dual-slot-ab-architecture)
|
||||||
|
4. [Technical Documentation Index](#4-technical-documentation-index)
|
||||||
|
5. [Repository Structure](#5-repository-structure)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Quick Start
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
* Linux, macOS, or Windows
|
||||||
|
* Python 3.8+
|
||||||
|
* `7z` (p7zip) or Python `py7zr` (`pip install -r requirements.txt`)
|
||||||
|
|
||||||
|
### Generate Installer Images
|
||||||
|
|
||||||
|
Build for **Ender-3 V3 KE** (default):
|
||||||
|
```bash
|
||||||
|
./build_openke_image.sh --target ke
|
||||||
|
```
|
||||||
|
|
||||||
|
Build for **Nebula Pad**:
|
||||||
|
```bash
|
||||||
|
./build_openke_image.sh --target nebula
|
||||||
|
```
|
||||||
|
|
||||||
|
Build for **Both Devices in One Run**:
|
||||||
|
```bash
|
||||||
|
./build_openke_image.sh --target all
|
||||||
|
```
|
||||||
|
|
||||||
|
By default, the script automatically:
|
||||||
|
1. Locates OpenKE build artifacts in `../OpenKE/artifacts/buildroot-halley5-v30-image/`.
|
||||||
|
2. Validates partition budgets:
|
||||||
|
- `xImage`: 5.54 MiB / 8 MiB (66.1% capacity, 2.71 MiB free).
|
||||||
|
- `rootfs.squashfs`: 330.85 MiB / 500 MiB (66.2% capacity, 169.15 MiB free).
|
||||||
|
3. Verifies SHA256 checksums against `build-manifest.txt` (if present).
|
||||||
|
4. Slices payloads into 1 MiB chunks with Creality MD5 sidecars.
|
||||||
|
5. Encrypts the 7z container with the board-derived Creality key (`$1$cxswfile$...`).
|
||||||
|
6. Verifies archive integrity and outputs the `.img` to `build/`.
|
||||||
|
|
||||||
|
### Custom Arguments & CLI Flags
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install -r requirements.txt
|
# Custom artifacts directory and custom version
|
||||||
|
./build_openke_image.sh --target nebula --artifacts /path/to/artifacts --version 1.1.0.35
|
||||||
|
|
||||||
|
# Or invoke the Python packager directly
|
||||||
|
python3 tools/pack_openke.py --target nebula --artifacts-dir ../OpenKE/artifacts/buildroot-halley5-v30-image
|
||||||
```
|
```
|
||||||
|
|
||||||
On Linux, install `squashfs-tools-ng` so `gensquashfs` and `rdsquashfs` are
|
---
|
||||||
available on `PATH`. On Windows, the helper uses the bundled `.exe` tools.
|
|
||||||
|
|
||||||
## Rebuild flow
|
## 2. Flashing Guide
|
||||||
|
|
||||||
|
1. Format a USB flash drive as **FAT32** with an **MBR** (Master Boot Record) partition table.
|
||||||
|
2. Copy the generated `.img` file to the **root** of the USB drive:
|
||||||
|
- For Ender-3 V3 KE: `Ender-3_V3_KE_F005_ota_img_V1.1.0.34.img`
|
||||||
|
- For Nebula Pad: `NEBULA_ota_img_V1.1.0.34.img`
|
||||||
|
3. Insert the USB drive into the front USB port of the printer / Nebula Pad.
|
||||||
|
4. On the touchscreen: Navigate to **Settings** $\rightarrow$ **Firmware Update** $\rightarrow$ **Local Update**.
|
||||||
|
5. Select the firmware image and confirm the update.
|
||||||
|
6. The stock updater will:
|
||||||
|
- Flash `xImage` to `kernel2` (`/dev/mmcblk0p6`).
|
||||||
|
- Flash `rootfs.squashfs` to `rootfs2` (`/dev/mmcblk0p8`).
|
||||||
|
- Flash `zero.bin` to `rtos2` (`/dev/mmcblk0p4`).
|
||||||
|
- Set the bootloader marker in `/dev/mmcblk0p1` to `ota:kernel2`.
|
||||||
|
- Automatically reboot into OpenKE.
|
||||||
|
7. On first boot, OpenKE seeds `/usr/data/openke/` and starts Klipper, Moonraker, and GuppyScreen.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Safety & Dual-Slot A/B Architecture
|
||||||
|
|
||||||
|
* **Non-Destructive Install**: The stock CrealityOS operating system resides permanently in **Slot 1** (`mmcblk0p5`/`p7`) and is **never overwritten**. OpenKE is installed cleanly into **Slot 2** (`mmcblk0p6`/`p8`).
|
||||||
|
* **Persistent Data Isolation**: Both operating systems share `/usr/data`, but OpenKE isolates its entire runtime in `/usr/data/openke/`. Stock Creality configs (`/usr/data/creality/`), printer calibrations, and WiFi credentials remain completely untouched.
|
||||||
|
* **Instant Reversion to Stock**: If you ever want to revert to stock CrealityOS, simply toggle the boot marker back to Slot 1:
|
||||||
|
```bash
|
||||||
|
# From OpenKE SSH shell:
|
||||||
|
echo -n "ota:kernel" > /dev/mmcblk0p1 && sync && reboot
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Technical Documentation Index
|
||||||
|
|
||||||
|
For in-depth architectural and developer documentation:
|
||||||
|
|
||||||
|
* **[Creality OTA Packaging Specification](docs/CREALITY_OTA_SPECIFICATION.md)**:
|
||||||
|
Full reverse-engineered specification of Creality's 7z container, MD5-crypt key derivation, 1 MiB payload chunking, manifest formats, and updater execution flow.
|
||||||
|
* **[Hardware A/B Partition Model](docs/A_B_PARTITION_MODEL.md)**:
|
||||||
|
Physical partition table, partition capacities, bootloader marker mechanics, and `/usr/data` namespace isolation.
|
||||||
|
* **[Troubleshooting & Diagnostics](docs/TROUBLESHOOTING.md)**:
|
||||||
|
Common USB update pitfalls, drive formatting, version string parsing, and how to inspect live updater logs on the device.
|
||||||
|
* **[Firmware Notes](FIRMWARE_NOTES.md)**:
|
||||||
|
Official baselines, derived encryption keys, partition budgets, and tool usage notes.
|
||||||
|
* **[Firmware Changelog](FIRMWARE_CHANGELOG.md)**:
|
||||||
|
Project history from DarKE v0.0.1 pre-root to OpenKE USB Installer v1.0.0.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Repository Structure
|
||||||
|
|
||||||
```powershell
|
|
||||||
python .\tools\ke_firmware.py build --stock-archive .\Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img --overlay-dir .\overlay_rootfs --output .\build\DarKE_KE_F005_ota_img_V11034.img
|
|
||||||
```
|
```
|
||||||
|
DarKE/
|
||||||
If the stock archive is missing, pass `--stock-url` and the helper will download it before extracting.
|
├── Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img # Stock CrealityOS baseline for Ender-3 V3 KE (F005)
|
||||||
If your `C:` drive is tight, point `--build-dir` at a larger drive so the stock rootfs can be unpacked there.
|
├── NEBULA_ota_img_V1.1.0.30.img # Stock CrealityOS baseline for Nebula Pad (NEBULA)
|
||||||
|
├── build_openke_image.sh # Automated build wrapper script
|
||||||
The build step downloads or reuses the stock archive, extracts it, overlays `overlay_rootfs/`, rebuilds the root filesystem with the Windows `squashfs-tools-ng` binaries, and repacks the Creality image envelope with `py7zr`.
|
├── README.md # Workspace overview & quickstart
|
||||||
The rebuild is metadata-preserving, so stock symlinks and executable bits survive the round trip.
|
├── FIRMWARE_NOTES.md # Architecture notes & encryption keys
|
||||||
|
├── FIRMWARE_CHANGELOG.md # Version changelog & release milestones
|
||||||
## Compare flow
|
├── requirements.txt # Python dependencies (`py7zr`)
|
||||||
|
├── assets/ # Standalone RTOS binaries
|
||||||
```powershell
|
│ ├── zero.bin # Stock RTOS for Ender-3 V3 KE (F005)
|
||||||
python .\tools\ke_firmware.py compare --stock-archive .\Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img --rebuilt-archive .\build\DarKE_KE_F005_ota_img_V11034.img --overlay-dir .\overlay_rootfs
|
│ └── zero_nebula.bin # Stock RTOS for Nebula Pad (NEBULA)
|
||||||
|
├── docs/ # In-depth technical documentation
|
||||||
|
│ ├── CREALITY_OTA_SPECIFICATION.md # Creality OTA container & updater internals
|
||||||
|
│ ├── A_B_PARTITION_MODEL.md # GPT partition layout & slot mechanics
|
||||||
|
│ └── TROUBLESHOOTING.md # USB diagnostics & common issues
|
||||||
|
├── tools/
|
||||||
|
│ └── pack_openke.py # OpenKE stock OTA packager
|
||||||
|
└── build/ # Generated OTA installer images
|
||||||
|
├── Ender-3_V3_KE_F005_ota_img_V1.1.0.34.img
|
||||||
|
└── NEBULA_ota_img_V1.1.0.34.img
|
||||||
```
|
```
|
||||||
|
|
||||||
Use this after a build to confirm the rebuilt rootfs still matches stock everywhere except the overlayed files.
|
|
||||||
|
|
||||||
## Stock flash flow
|
|
||||||
|
|
||||||
1. Copy the generated `.img` to a USB drive.
|
|
||||||
2. Insert the USB drive into the Ender-3 V3 KE.
|
|
||||||
3. Use the built-in updater to flash the image.
|
|
||||||
4. After reboot, the firmware reports the UI/display version from the package metadata.
|
|
||||||
|
|
||||||
This flow does not require manual edits on the printer itself.
|
|
||||||
|
|||||||
Binary file not shown.
Binary file not shown.
Executable
+147
@@ -0,0 +1,147 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
#
|
||||||
|
# build_openke_image.sh
|
||||||
|
#
|
||||||
|
# Generates a stock-compatible CrealityOS OTA firmware (.img) directly from
|
||||||
|
# OpenKE build artifacts. The resulting file can be placed on a FAT32 USB drive
|
||||||
|
# and flashed via the stock touchscreen UI to install OpenKE to Slot 2.
|
||||||
|
#
|
||||||
|
# Supports:
|
||||||
|
# - Ender-3 V3 KE (Target F005)
|
||||||
|
# - Nebula Smart Kit / Nebula Pad (Target NEBULA)
|
||||||
|
#
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
cd "${SCRIPT_DIR}"
|
||||||
|
|
||||||
|
# Defaults
|
||||||
|
DEFAULT_ARTIFACTS="${SCRIPT_DIR}/../OpenKE/artifacts/buildroot-halley5-v30-image"
|
||||||
|
DEFAULT_VERSION="1.1.0.34"
|
||||||
|
DEFAULT_TARGET="ke"
|
||||||
|
|
||||||
|
ARTIFACTS_DIR="${DEFAULT_ARTIFACTS}"
|
||||||
|
VERSION="${DEFAULT_VERSION}"
|
||||||
|
TARGET="${DEFAULT_TARGET}"
|
||||||
|
OUTPUT=""
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat << EOF
|
||||||
|
Usage: $(basename "$0") [OPTIONS]
|
||||||
|
|
||||||
|
Packages OpenKE kernel and rootfs into a stock CrealityOS-supported update .img.
|
||||||
|
|
||||||
|
Options:
|
||||||
|
-t, --target TARGET Target hardware profile:
|
||||||
|
ke - Ender-3 V3 KE (F005) [Default]
|
||||||
|
nebula - Creality Nebula Pad (NEBULA)
|
||||||
|
all - Build images for both devices
|
||||||
|
-a, --artifacts DIR Directory containing OpenKE build artifacts (xImage, rootfs.squashfs)
|
||||||
|
(Default: ../OpenKE/artifacts/buildroot-halley5-v30-image)
|
||||||
|
-v, --version VER Package version for Creality updater (Default: 1.1.0.34)
|
||||||
|
-o, --output FILE Output .img file path (single target only)
|
||||||
|
-h, --help Show this help message and exit
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
./build_openke_image.sh # Builds for Ender-3 V3 KE
|
||||||
|
./build_openke_image.sh --target nebula # Builds for Nebula Pad
|
||||||
|
./build_openke_image.sh --target all # Builds both KE and Nebula images
|
||||||
|
EOF
|
||||||
|
exit 0
|
||||||
|
}
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
-t|--target)
|
||||||
|
TARGET="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-a|--artifacts)
|
||||||
|
ARTIFACTS_DIR="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-v|--version)
|
||||||
|
VERSION="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-o|--output)
|
||||||
|
OUTPUT="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
usage
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "Unknown option: $1" >&2
|
||||||
|
usage
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "=================================================================="
|
||||||
|
echo " OpenKE Stock OTA Firmware Generator"
|
||||||
|
echo "=================================================================="
|
||||||
|
echo " Target Device: ${TARGET}"
|
||||||
|
echo " Artifacts Directory: ${ARTIFACTS_DIR}"
|
||||||
|
echo " Package Version: ${VERSION}"
|
||||||
|
if [[ -n "${OUTPUT}" ]]; then
|
||||||
|
echo " Custom Output: ${OUTPUT}"
|
||||||
|
fi
|
||||||
|
echo "=================================================================="
|
||||||
|
|
||||||
|
# Check Python3
|
||||||
|
if ! command -v python3 &>/dev/null; then
|
||||||
|
echo "ERROR: python3 is not installed or not in PATH." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Ensure assets are present
|
||||||
|
mkdir -p "${SCRIPT_DIR}/assets"
|
||||||
|
if [[ ! -f "${SCRIPT_DIR}/assets/zero.bin" && -f "${SCRIPT_DIR}/Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img" ]]; then
|
||||||
|
echo "[*] Extracting assets/zero.bin from KE stock image..."
|
||||||
|
7z e -p'$1$cxswfile$ZFd0RWFYkJQugbtKVGL9y0' -r "${SCRIPT_DIR}/Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img" 'zero.bin*' -o"${SCRIPT_DIR}/assets" -y >/dev/null
|
||||||
|
cp "${SCRIPT_DIR}/assets/zero.bin.0000."* "${SCRIPT_DIR}/assets/zero.bin" 2>/dev/null || true
|
||||||
|
rm -f "${SCRIPT_DIR}/assets/zero.bin.0000."*
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -f "${SCRIPT_DIR}/assets/zero_nebula.bin" && -f "${SCRIPT_DIR}/NEBULA_ota_img_V1.1.0.30.img" ]]; then
|
||||||
|
echo "[*] Extracting assets/zero_nebula.bin from Nebula stock image..."
|
||||||
|
7z e -p'$1$cxswfile$XG7ANbYX3bq2H2xmmwrka.' -r "${SCRIPT_DIR}/NEBULA_ota_img_V1.1.0.30.img" 'zero.bin*' -o"${SCRIPT_DIR}/assets" -y >/dev/null
|
||||||
|
cp "${SCRIPT_DIR}/assets/zero.bin.0000."* "${SCRIPT_DIR}/assets/zero_nebula.bin" 2>/dev/null || true
|
||||||
|
rm -f "${SCRIPT_DIR}/assets/zero.bin.0000."*
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check OpenKE artifacts
|
||||||
|
if [[ ! -d "${ARTIFACTS_DIR}" ]]; then
|
||||||
|
echo "ERROR: Artifacts directory not found: ${ARTIFACTS_DIR}" >&2
|
||||||
|
echo "Please build OpenKE first or specify --artifacts <dir>." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -f "${ARTIFACTS_DIR}/xImage" ]]; then
|
||||||
|
echo "ERROR: ${ARTIFACTS_DIR}/xImage does not exist." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ ! -f "${ARTIFACTS_DIR}/rootfs.squashfs" ]]; then
|
||||||
|
echo "ERROR: ${ARTIFACTS_DIR}/rootfs.squashfs does not exist." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Build arguments
|
||||||
|
ARGS=(
|
||||||
|
"--target" "${TARGET}"
|
||||||
|
"--artifacts-dir" "${ARTIFACTS_DIR}"
|
||||||
|
"--version" "${VERSION}"
|
||||||
|
)
|
||||||
|
|
||||||
|
if [[ -n "${OUTPUT}" ]]; then
|
||||||
|
ARGS+=("--output" "${OUTPUT}")
|
||||||
|
fi
|
||||||
|
|
||||||
|
python3 "${SCRIPT_DIR}/tools/pack_openke.py" "${ARGS[@]}"
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "Build complete! Output files are located in build/"
|
||||||
|
ls -lh "${SCRIPT_DIR}/build/"
|
||||||
@@ -0,0 +1,86 @@
|
|||||||
|
# Hardware Partition Layout & Dual-Slot A/B Boot Model
|
||||||
|
|
||||||
|
This document details the eMMC physical partition layout, the hardware A/B slot mechanics, and the persistent storage architecture used by Creality X2000 devices (Ender-3 V3 KE and Nebula Pad).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Physical Partition Table
|
||||||
|
|
||||||
|
The onboard eMMC is partitioned with a standard GPT partition table:
|
||||||
|
|
||||||
|
| Partition | Label | Fixed Capacity | Mount / Role | Active Slot 1 (Stock) | Active Slot 2 (OpenKE) |
|
||||||
|
| :--- | :--- | :--- | :--- | :--- | :--- |
|
||||||
|
| `/dev/mmcblk0p1` | `ota` | **1 MiB** | Boot marker storage | Stores `ota:kernel` or `ota:kernel2` | Shared |
|
||||||
|
| `/dev/mmcblk0p2` | `sn_mac` | **1 MiB** | Factory serial & MAC addresses | Read-only identity | Shared |
|
||||||
|
| `/dev/mmcblk0p3` | `rtos` | **4 MiB** | Aux RTOS firmware (Slot 1) | Active | Inactive |
|
||||||
|
| `/dev/mmcblk0p4` | `rtos2` | **4 MiB** | Aux RTOS firmware (Slot 2) | Inactive | Active |
|
||||||
|
| `/dev/mmcblk0p5` | `kernel` | **8 MiB** | Linux uImage (Slot 1) | Active | Inactive |
|
||||||
|
| `/dev/mmcblk0p6` | `kernel2` | **8 MiB** | Linux uImage (Slot 2) | Inactive | Active |
|
||||||
|
| `/dev/mmcblk0p7` | `rootfs` | **500 MiB** | SquashFS rootfs (Slot 1) | Active | Inactive |
|
||||||
|
| `/dev/mmcblk0p8` | `rootfs2` | **500 MiB** | SquashFS rootfs (Slot 2) | Inactive | Active |
|
||||||
|
| `/dev/mmcblk0p9` | `rootfs_data` | **300 MiB** | Ext4 writable overlay | `/overlay` | `/overlay` |
|
||||||
|
| `/dev/mmcblk0p10` | `userdata` | **~6.1 GiB** | Ext4 persistent storage | Mounted at `/usr/data` | Mounted at `/usr/data` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Boot Selection via the OTA Marker
|
||||||
|
|
||||||
|
The bootloader (U-Boot) reads the 1 MiB `ota` partition (`/dev/mmcblk0p1`) on power-up to decide which kernel and rootfs to mount. The partition contains a simple ASCII string:
|
||||||
|
|
||||||
|
* `ota:kernel` $\rightarrow$ Boots **Slot 1** (Stock CrealityOS: kernel `p5`, rootfs `p7`).
|
||||||
|
* `ota:kernel2` $\rightarrow$ Boots **Slot 2** (Custom / OpenKE: kernel `p6`, rootfs `p8`).
|
||||||
|
|
||||||
|
### How Stock OTA Updater Flashes to Slot 2
|
||||||
|
|
||||||
|
When you run a USB update from the stock Creality touchscreen:
|
||||||
|
1. Stock firmware queries which slot is currently running (`local_get_kernel_dev_path`).
|
||||||
|
2. Since the printer is running on Slot 1 (`ota:kernel`), the updater targets the **inactive** slot:
|
||||||
|
- Flashes `xImage` to `kernel2` (`mmcblk0p6`).
|
||||||
|
- Flashes `rootfs.squashfs` to `rootfs2` (`mmcblk0p8`).
|
||||||
|
- Flashes `zero.bin` to `rtos2` (`mmcblk0p4`).
|
||||||
|
3. Calls `local_set_next_boot_device`, which writes `ota:kernel2` into `mmcblk0p1`.
|
||||||
|
4. Reboots into OpenKE.
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> Stock firmware in Slot 1 (`mmcblk0p5` and `mmcblk0p7`) is **never overwritten** by this process.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Persistent Data Isolation (`/usr/data`)
|
||||||
|
|
||||||
|
`/dev/mmcblk0p10` is the single shared ~6.1 GiB persistent partition mounted at `/usr/data`. Both Slot 1 and Slot 2 share this partition, but they use strict directory separation:
|
||||||
|
|
||||||
|
```
|
||||||
|
/usr/data/
|
||||||
|
├── creality/ # Stock CrealityOS files (gcode, logs, userdata, factory data)
|
||||||
|
│ ├── userdata/
|
||||||
|
│ └── printer_data/
|
||||||
|
└── openke/ # OpenKE isolated runtime
|
||||||
|
├── apps/ # Mutable checkouts (klipper, moonraker, mainsail)
|
||||||
|
├── envs/ # Python virtual environments
|
||||||
|
├── system/ # Boot logs, activation state, machine configs
|
||||||
|
└── printer_data/ # Klipper config (printer.cfg), macros, Moonraker DB
|
||||||
|
```
|
||||||
|
|
||||||
|
Because OpenKE namespaces all its data under `/usr/data/openke/`, flashing OpenKE does not erase or corrupt stock print history, WiFi credentials, or factory calibration files.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Reverting to Stock Firmware
|
||||||
|
|
||||||
|
If you ever wish to return to stock CrealityOS, you do not need to reflash. You simply toggle the boot marker back to Slot 1:
|
||||||
|
|
||||||
|
### From OpenKE Shell (SSH)
|
||||||
|
```bash
|
||||||
|
echo -n "ota:kernel" > /dev/mmcblk0p1
|
||||||
|
sync
|
||||||
|
reboot
|
||||||
|
```
|
||||||
|
|
||||||
|
### From Stock CrealityOS (if toggling forward again)
|
||||||
|
```bash
|
||||||
|
. /etc/ota_bin/ota_utils.sh
|
||||||
|
. /etc/ota_bin/ota_local_method.sh
|
||||||
|
local_set_next_boot_device
|
||||||
|
reboot
|
||||||
|
```
|
||||||
@@ -0,0 +1,168 @@
|
|||||||
|
# CrealityOS OTA Packaging & Update Subsystem Specification
|
||||||
|
|
||||||
|
This document provides a technical specification of Creality's Over-The-Air (OTA) firmware update container format, encryption model, payload chunking, and the on-device update execution engine used across the **Ender-3 V3 KE** (board target `F005`) and the **Nebula Smart Kit / Nebula Pad** (board target `NEBULA`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Overview & Security Model
|
||||||
|
|
||||||
|
Creality's local (USB) and network OTA update mechanism on Ingenic X2000 devices is **integrity-checked but unauthenticated**:
|
||||||
|
|
||||||
|
* **Archive Envelope**: A password-protected 7z archive with encrypted headers (`-mhe=on`).
|
||||||
|
* **Encryption Key**: Derived deterministically from the hardware board name using standard Unix MD5 crypt.
|
||||||
|
* **Payload Verification**: Each sub-payload (`xImage`, `rootfs.squashfs`, `zero.bin`) is checked using MD5 checksums.
|
||||||
|
* **No Cryptographic Signatures**: The device does **not** enforce RSA digital signatures or public key verification during OTA updates. Unsigned custom kernels and root filesystems flash and boot cleanly through stock firmware.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Key Derivation Formula
|
||||||
|
|
||||||
|
The 7z archive password is generated per board type using standard Unix MD5 password encryption (`$1$` format) with a hardcoded salt:
|
||||||
|
|
||||||
|
$$\text{Password} = \text{MD5-Crypt}(\text{Key} = \text{BOARD\_NAME} + \text{"C3\_7e\_bz"}, \text{Salt} = \text{"cxswfile"})$$
|
||||||
|
|
||||||
|
In shell (using `openssl`):
|
||||||
|
```bash
|
||||||
|
openssl passwd -1 -salt cxswfile "${BOARD_NAME}C3_7e_bz"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Known Board Encryption Keys
|
||||||
|
|
||||||
|
| Target Device | Board Identifier | Input String | Resulting 7z Password |
|
||||||
|
| :--- | :--- | :--- | :--- |
|
||||||
|
| **Ender-3 V3 KE** | `F005` | `F005C3_7e_bz` | `$1$cxswfile$ZFd0RWFYkJQugbtKVGL9y0` |
|
||||||
|
| **Nebula Pad** | `NEBULA` | `NEBULAC3_7e_bz` | `$1$cxswfile$XG7ANbYX3bq2H2xmmwrka.` |
|
||||||
|
| **Creality K1** | `CR4CU220812S12` | `CR4CU220812S12C3_7e_bz` | Board-derived key |
|
||||||
|
|
||||||
|
> [!TIP]
|
||||||
|
> `tools/pack_openke.py` includes a pure-Python implementation of Unix MD5-crypt, making password derivation completely cross-platform without external tools.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Archive Structure & Layout
|
||||||
|
|
||||||
|
Creality's update extractor (`/etc/ota_bin/local_ota_update.sh`) has strict structural expectations.
|
||||||
|
|
||||||
|
### Directory Hierarchy
|
||||||
|
|
||||||
|
For an archive file named `<ARCHIVE_STEM>.img` with package version `<VERSION>` (e.g. `NEBULA_ota_img_V1.1.0.34.img` with version `1.1.0.34`):
|
||||||
|
|
||||||
|
```
|
||||||
|
<ARCHIVE_STEM>/ # Top-level directory inside 7z
|
||||||
|
├── ota_config.in # Root version pointer
|
||||||
|
└── ota_v<VERSION>/ # Versioned payload folder
|
||||||
|
├── ota_v<VERSION>.ok # Readiness marker (touch file)
|
||||||
|
├── ota_update.in # Primary payload manifest
|
||||||
|
├── ota_md5_xImage.<FULL_MD5> # Chunk MD5 sidecar for kernel
|
||||||
|
├── ota_md5_rootfs.squashfs.<FULL_MD5> # Chunk MD5 sidecar for rootfs
|
||||||
|
├── ota_md5_zero.bin.<FULL_MD5> # Chunk MD5 sidecar for RTOS
|
||||||
|
├── xImage.0000.<FULL_MD5> # 1 MiB chunk 0
|
||||||
|
├── xImage.0001.<CHUNK0_MD5> # 1 MiB chunk 1
|
||||||
|
├── ...
|
||||||
|
├── rootfs.squashfs.0000.<FULL_MD5>
|
||||||
|
├── ...
|
||||||
|
└── zero.bin.0000.<FULL_MD5>
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Manifest Formats
|
||||||
|
|
||||||
|
### 1. `ota_config.in`
|
||||||
|
Located at the root of the extracted folder:
|
||||||
|
```ini
|
||||||
|
current_version=1.1.0.34
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. `ota_v<VERSION>.ok`
|
||||||
|
An empty 0-byte or 1-byte newline file placed in `ota_v<VERSION>/`. If missing, the updater aborts.
|
||||||
|
|
||||||
|
### 3. `ota_update.in`
|
||||||
|
Lists each payload with its partition type, internal filename, byte size, and full-file MD5:
|
||||||
|
```ini
|
||||||
|
ota_version=1.1.0.34
|
||||||
|
|
||||||
|
img_type=kernel
|
||||||
|
img_name=xImage
|
||||||
|
img_size=5541952
|
||||||
|
img_md5=f3bf93ad2fd1e770dacb3062ba9cf05c
|
||||||
|
|
||||||
|
img_type=rootfs
|
||||||
|
img_name=rootfs.squashfs
|
||||||
|
img_size=346923008
|
||||||
|
img_md5=7d0d3742dedd7a236632e4f7439f52dd
|
||||||
|
|
||||||
|
img_type=rtos
|
||||||
|
img_name=zero.bin
|
||||||
|
img_size=424264
|
||||||
|
img_md5=8e852ae9b8f5b78f3818e4236c3e8f1d
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Sidecar Checksum Files (`ota_md5_<NAME>.<FULL_MD5>`)
|
||||||
|
Text files containing the MD5 checksum of each 1 MiB chunk in order, separated by newlines:
|
||||||
|
```
|
||||||
|
e10adc3949ba59abbe56e057f20f883e
|
||||||
|
c33367701511b4f6020ec61ded352059
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Payload Slicing & Chunking Algorithm
|
||||||
|
|
||||||
|
Creality splits large image files into fixed **1 MiB** (1,048,576 bytes) chunks to manage RAM constraints on the 208MB-RAM device during download and flashing:
|
||||||
|
|
||||||
|
1. **Chunk Naming Convention**:
|
||||||
|
$$\text{chunk\_filename} = \text{base\_name} + \text{"."} + \text{index:04d} + \text{"."} + \text{prev\_md5}$$
|
||||||
|
2. **First Chunk (`index = 0000`)**:
|
||||||
|
`prev_md5` is set to the **FULL MD5** of the unchunked source file.
|
||||||
|
3. **Subsequent Chunks (`index > 0000`)**:
|
||||||
|
`prev_md5` is the MD5 checksum of the **immediately preceding chunk** (`index - 1`).
|
||||||
|
4. **Final Chunk**:
|
||||||
|
Carries the remaining bytes ($\le 1,048,576$ bytes).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. On-Device Update Execution Flow
|
||||||
|
|
||||||
|
Understanding the device-side execution path in `/etc/ota_bin/local_ota_update.sh` explains the strict naming rules:
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
autonumber
|
||||||
|
actor User as User via Touchscreen
|
||||||
|
participant MS as master-server (UI)
|
||||||
|
participant LU as local_ota_update.sh
|
||||||
|
participant OF as ota_file (7z Decryptor)
|
||||||
|
participant BL as Block Devices (mmcblk0)
|
||||||
|
|
||||||
|
User->>MS: Selects USB Update
|
||||||
|
MS->>MS: Scans for 'ota_img_V' on USB
|
||||||
|
MS->>MS: Compares versions (regex [0-9]{1,3})
|
||||||
|
MS->>LU: Calls local_ota_update.sh /media/sda1/<NAME>.img
|
||||||
|
LU->>LU: Computes OTA_UNZIP_FILE_NAME=${OTA_FILE_NAME%.img*}
|
||||||
|
LU->>OF: Calls ota_file e <NAME>.img <TEMP_DIR>
|
||||||
|
OF->>OF: Derives key & extracts 7z archive
|
||||||
|
LU->>LU: Checks <TEMP_DIR>/<OTA_UNZIP_FILE_NAME>/ota_config.in
|
||||||
|
LU->>LU: Validates partition sizes against device nodes
|
||||||
|
LU->>BL: Streams xImage chunks -> mmcblk0p6 (kernel2)
|
||||||
|
LU->>BL: Streams rootfs chunks -> mmcblk0p8 (rootfs2)
|
||||||
|
LU->>BL: Streams zero.bin chunks -> mmcblk0p4 (rtos2)
|
||||||
|
LU->>BL: Writes "ota:kernel2" to /dev/mmcblk0p1
|
||||||
|
LU->>MS: Returns success
|
||||||
|
MS->>User: Reboots printer into OpenKE
|
||||||
|
```
|
||||||
|
|
||||||
|
### The Critical Folder-Name Matching Rule
|
||||||
|
|
||||||
|
Lines 312–320 of `local_ota_update.sh`:
|
||||||
|
```bash
|
||||||
|
OTA_FILE_NAME=$(basename ${OTA_FILE})
|
||||||
|
OTA_UNZIP_FILE_NAME=${OTA_FILE_NAME%.img*}
|
||||||
|
ota_site=${OTA_FILE_PATH}/${OTA_UNZIP_FILE_NAME}
|
||||||
|
/usr/bin/ota_file e ${OTA_FILE_PATH}/${OTA_FILE_NAME} ${OTA_FILE_PATH}
|
||||||
|
ota_site_config=$ota_site/ota_config.in
|
||||||
|
```
|
||||||
|
|
||||||
|
> [!IMPORTANT]
|
||||||
|
> If the archive file is named `NEBULA_ota_img_V1.1.0.34.img`, `OTA_UNZIP_FILE_NAME` evaluates to `NEBULA_ota_img_V1.1.0.34`. The root directory inside the 7z archive **must match that exact string**. If the archive has a different folder name (e.g. `OpenKE_...`), `cp -f $ota_site_config` fails and the update aborts immediately.
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
# Troubleshooting & USB Flashing Diagnostics
|
||||||
|
|
||||||
|
This document covers common issues when flashing OpenKE via USB OTA, root causes, and diagnostic steps.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. "No firmware found" / "No local update available"
|
||||||
|
|
||||||
|
When you insert the USB drive and navigate to **Settings** $\rightarrow$ **Firmware Update** $\rightarrow$ **Local Update**, the screen reports no file found.
|
||||||
|
|
||||||
|
### Root Causes & Fixes:
|
||||||
|
1. **USB Drive File System**:
|
||||||
|
* The Creality Linux kernel does not reliably mount exFAT or NTFS drives on early boot.
|
||||||
|
* **Fix**: Format the USB flash drive as **FAT32** with an **MBR** (Master Boot Record) partition table. Avoid GPT partition tables on the flash drive.
|
||||||
|
2. **File Location**:
|
||||||
|
* The `.img` file must be placed in the **root directory** of the USB drive (e.g. `E:\NEBULA_ota_img_V1.1.0.34.img`), not inside a folder.
|
||||||
|
3. **Filename Format**:
|
||||||
|
* Creality's `master-server` scans for files matching `ota_img_V`.
|
||||||
|
* Files must be named:
|
||||||
|
- For Nebula Pad: `NEBULA_ota_img_V<version>.img`
|
||||||
|
- For Ender-3 V3 KE: `Ender-3_V3_KE_F005_ota_img_V<version>.img`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. "Update failed" Immediately After Confirming
|
||||||
|
|
||||||
|
The screen detects the update, you click "Update Now", the progress bar appears for a split second, and it immediately reports "Update Failed".
|
||||||
|
|
||||||
|
### Root Cause:
|
||||||
|
* Creality's updater extracts the 7z archive to a temporary directory and looks for `ota_config.in` in `<TEMP_DIR>/<UNZIP_FOLDER>/ota_config.in`.
|
||||||
|
* If the root directory inside the 7z archive does not match the `.img` filename (without `.img`), `cp -f $ota_site_config` fails.
|
||||||
|
* **Resolution**: The updated `build_openke_image.sh` / `tools/pack_openke.py` guarantees this alignment automatically. Ensure you use images built with the latest packager.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. "Current version is newest" / Won't Allow Flashing
|
||||||
|
|
||||||
|
The UI says the firmware is already up-to-date or newer.
|
||||||
|
|
||||||
|
### Root Cause:
|
||||||
|
* In `local_ota_update.sh`:
|
||||||
|
```bash
|
||||||
|
if [ $current_version -ge $ota_current_version ]; then
|
||||||
|
echo "ota not update: current version is newest"
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
```
|
||||||
|
* If your device is running `1.1.0.30` and the package version is `1.1.0.30` or lower, it refuses to flash.
|
||||||
|
* **Fix**: Pass a higher version number when running the build script:
|
||||||
|
```bash
|
||||||
|
./build_openke_image.sh --target nebula --version 1.1.0.35
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. How to Inspect Live Update Logs on the Printer
|
||||||
|
|
||||||
|
If you have SSH access to stock firmware and want to see what `local_ota_update.sh` is doing during a flash:
|
||||||
|
|
||||||
|
1. Connect to the printer via SSH:
|
||||||
|
```bash
|
||||||
|
ssh root@<printer-ip>
|
||||||
|
```
|
||||||
|
2. Trigger the local update manually from the command line to see stdout/stderr in real time:
|
||||||
|
```bash
|
||||||
|
sh -x /etc/ota_bin/local_ota_update.sh /media/sda1/NEBULA_ota_img_V1.1.0.34.img
|
||||||
|
```
|
||||||
|
3. Check temporary updater logs:
|
||||||
|
```bash
|
||||||
|
ls -la /usr/data/creality/ota_updater*
|
||||||
|
```
|
||||||
@@ -1 +0,0 @@
|
|||||||
etc/init.d/S98sync_version
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
|
|
||||||
AGREE_ROOT_FILE=/usr/data/creality/userdata/user_agree_root
|
|
||||||
|
|
||||||
case "$1" in
|
|
||||||
start)
|
|
||||||
mkdir -p /usr/data/creality/userdata
|
|
||||||
: > "$AGREE_ROOT_FILE"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
true
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
ota_version=1.1.0.33
|
|
||||||
ota_board_name=F005
|
|
||||||
ota_compile_time=2024 08.06 09:12:56
|
|
||||||
ota_site=http://192.168.43.52/ota/board_test
|
|
||||||
@@ -1,256 +0,0 @@
|
|||||||
# Ender-3V3KE
|
|
||||||
# Printer_size: 220x220x240
|
|
||||||
# Version: v1.1.0.33
|
|
||||||
# CreateDate: 2024/3/27
|
|
||||||
# mcu: chip: GD32F303RET6
|
|
||||||
# version: CR4NS200323C10
|
|
||||||
[include sensorless.cfg]
|
|
||||||
[include gcode_macro.cfg]
|
|
||||||
[include printer_params.cfg]
|
|
||||||
[mcu]
|
|
||||||
serial:/dev/ttyS1
|
|
||||||
baud:230400
|
|
||||||
restart_method: command
|
|
||||||
|
|
||||||
[force_move]
|
|
||||||
enable_force_move: True
|
|
||||||
|
|
||||||
[mcu rpi]
|
|
||||||
serial: /tmp/klipper_host_mcu
|
|
||||||
|
|
||||||
[bl24c16f]
|
|
||||||
i2c_mcu: rpi
|
|
||||||
i2c_bus: i2c.2
|
|
||||||
i2c_speed: 400000
|
|
||||||
|
|
||||||
######################################################
|
|
||||||
[prtouch_v2]
|
|
||||||
pres_cnt: 1 #探点次数
|
|
||||||
pres0_clk_pins: PA4 #压力检测时钟引脚配置
|
|
||||||
pres0_sdo_pins: PC6 #压力检测数据引脚配置
|
|
||||||
step_swap_pin: PA15
|
|
||||||
pres_swap_pin: PA15
|
|
||||||
step_base:2
|
|
||||||
# show_msg: True
|
|
||||||
tri_min_hold: 1000
|
|
||||||
tri_max_hold: 1500 #压力检测信息展示
|
|
||||||
speed: 1
|
|
||||||
# tri_wave_ip: 172.22.30.204
|
|
||||||
#####################################################
|
|
||||||
|
|
||||||
|
|
||||||
[z_compensate]
|
|
||||||
tri_min_hold: 1400
|
|
||||||
tri_max_hold: 2000 #压力检测信息展示
|
|
||||||
tri_expand_mm = 0.10
|
|
||||||
# tri_min_hold: 3
|
|
||||||
speed: 5
|
|
||||||
hot_start_temp: 180#擦喷嘴是最小温度
|
|
||||||
hot_rub_temp: 200#擦喷嘴是最小温度
|
|
||||||
hot_end_temp: 140#擦喷嘴是最小温度
|
|
||||||
bed_add_temp: 60#调平时的热床温度
|
|
||||||
clr_noz_start_x: -3 #擦喷头区域的起始x坐标(默认在热床后方正中心位置)
|
|
||||||
clr_noz_start_y: 20 #擦喷头区域的起始y坐标
|
|
||||||
clr_noz_len_x: 3 #擦喷头区域的x方向的长度
|
|
||||||
clr_noz_len_y: 50 #擦喷头区域的y方向的长度
|
|
||||||
pa_clr_dis_mm_x = 0
|
|
||||||
pa_clr_dis_mm_y =30
|
|
||||||
# show_msg = True
|
|
||||||
bl_offset: 0,27
|
|
||||||
noz_pos_center: 20,25
|
|
||||||
noz_pos_offset: 3,7
|
|
||||||
pumpback_mm: 10
|
|
||||||
vs_start_z_pos: 3
|
|
||||||
pr_probe_cnt: 3
|
|
||||||
pr_clear_probe_cnt: 3
|
|
||||||
type_nozz = 0
|
|
||||||
|
|
||||||
|
|
||||||
[printer]
|
|
||||||
kinematics: cartesian
|
|
||||||
max_velocity: 500
|
|
||||||
max_accel: 8000
|
|
||||||
max_accel_to_decel: 2500
|
|
||||||
max_z_velocity: 30
|
|
||||||
square_corner_velocity: 5.0
|
|
||||||
max_z_accel: 300
|
|
||||||
|
|
||||||
[idle_timeout]
|
|
||||||
timeout: 99999999
|
|
||||||
|
|
||||||
[stepper_x]
|
|
||||||
step_pin: PC2
|
|
||||||
dir_pin: !PB9
|
|
||||||
enable_pin: !PC3
|
|
||||||
microsteps: 16
|
|
||||||
rotation_distance: 40
|
|
||||||
endstop_pin: !PA5
|
|
||||||
position_endstop: -12
|
|
||||||
position_min: -12
|
|
||||||
position_max: 221
|
|
||||||
homing_speed: 30
|
|
||||||
homing_retract_dist:0 #10
|
|
||||||
|
|
||||||
[tmc2208 stepper_x]
|
|
||||||
uart_pin:PB12
|
|
||||||
interpolate: True
|
|
||||||
run_current:0.75
|
|
||||||
sense_resistor: 0.150
|
|
||||||
stealthchop_threshold: 0
|
|
||||||
|
|
||||||
|
|
||||||
[stepper_y]
|
|
||||||
step_pin: PB8
|
|
||||||
dir_pin: PB7
|
|
||||||
enable_pin: !PC3
|
|
||||||
microsteps: 16
|
|
||||||
rotation_distance: 60
|
|
||||||
endstop_pin: !PA6
|
|
||||||
position_endstop: -20
|
|
||||||
position_min: -20
|
|
||||||
position_max: 223
|
|
||||||
homing_speed: 30
|
|
||||||
homing_retract_dist:0
|
|
||||||
|
|
||||||
[tmc2208 stepper_y]
|
|
||||||
uart_pin:PB13
|
|
||||||
interpolate: True
|
|
||||||
run_current:0.75
|
|
||||||
sense_resistor: 0.150
|
|
||||||
stealthchop_threshold: 0
|
|
||||||
|
|
||||||
[stepper_z]
|
|
||||||
step_pin: PB6
|
|
||||||
dir_pin: !PB5
|
|
||||||
enable_pin: !PC3
|
|
||||||
microsteps: 16
|
|
||||||
rotation_distance:8
|
|
||||||
endstop_pin:probe:z_virtual_endstop#PA15 #probe:z_virtual_endstop
|
|
||||||
position_max: 246
|
|
||||||
position_min: -5
|
|
||||||
|
|
||||||
[tmc2208 stepper_z]
|
|
||||||
uart_pin: PB14
|
|
||||||
interpolate: True
|
|
||||||
run_current: 0.8
|
|
||||||
stealthchop_threshold: 0
|
|
||||||
sense_resistor: 0.150
|
|
||||||
|
|
||||||
|
|
||||||
[bltouch]
|
|
||||||
sensor_pin:PC14
|
|
||||||
control_pin: PC13
|
|
||||||
x_offset: 0
|
|
||||||
y_offset: 27
|
|
||||||
z_offset: 0
|
|
||||||
probe_with_touch_mode: true
|
|
||||||
stow_on_each_sample: false
|
|
||||||
speed:5
|
|
||||||
lift_speed:20
|
|
||||||
|
|
||||||
|
|
||||||
[filament_switch_sensor filament_sensor]
|
|
||||||
switch_pin: !PC15
|
|
||||||
pause_on_runout: true
|
|
||||||
|
|
||||||
[output_pin MainBoardFan]
|
|
||||||
pin: !PB1
|
|
||||||
|
|
||||||
[extruder]
|
|
||||||
max_extrude_only_distance:1000
|
|
||||||
max_extrude_cross_section:80
|
|
||||||
pressure_advance = 0.036
|
|
||||||
step_pin: PB4
|
|
||||||
dir_pin: PB3
|
|
||||||
enable_pin: !PC3
|
|
||||||
microsteps: 16
|
|
||||||
rotation_distance: 7.53
|
|
||||||
nozzle_diameter: 0.400
|
|
||||||
filament_diameter: 1.750
|
|
||||||
heater_pin: PA1
|
|
||||||
sensor_type: EPCOS 100K B57560G104F
|
|
||||||
sensor_pin: PC5
|
|
||||||
control = pid
|
|
||||||
pid_Kp=20.584
|
|
||||||
pid_Ki=1.737
|
|
||||||
pid_Kd=60.981
|
|
||||||
min_temp: 0
|
|
||||||
max_temp: 320 # Set to 300 for S1 Pro
|
|
||||||
|
|
||||||
|
|
||||||
[heater_bed]
|
|
||||||
heater_pin: PB2
|
|
||||||
sensor_type: EPCOS 100K B57560G104F
|
|
||||||
sensor_pin: PC4
|
|
||||||
control = pid
|
|
||||||
pid_kp = 70.652
|
|
||||||
pid_ki = 1.798
|
|
||||||
pid_kd = 694.157
|
|
||||||
min_temp: 0
|
|
||||||
max_temp: 120 #
|
|
||||||
temp_offset_flag = True
|
|
||||||
|
|
||||||
[verify_heater extruder]
|
|
||||||
[verify_heater heater_bed]
|
|
||||||
check_gain_time: 120
|
|
||||||
heating_gain: 1.0
|
|
||||||
hysteresis: 10
|
|
||||||
|
|
||||||
[temperature_sensor mcu_temp]
|
|
||||||
sensor_type: temperature_mcu
|
|
||||||
min_temp: 0
|
|
||||||
max_temp: 100
|
|
||||||
|
|
||||||
[output_pin fan0]
|
|
||||||
pin:PA0
|
|
||||||
pwm: True
|
|
||||||
cycle_time: 0.0100
|
|
||||||
hardware_pwm: false
|
|
||||||
value: 0.00
|
|
||||||
scale: 255
|
|
||||||
shutdown_value: 0.0
|
|
||||||
|
|
||||||
[heater_fan nozzle_fan]
|
|
||||||
pin: PC1
|
|
||||||
max_power: 1.0
|
|
||||||
shutdown_speed: 0
|
|
||||||
cycle_time: 0.010
|
|
||||||
hardware_pwm: False
|
|
||||||
kick_start_time: 0.100
|
|
||||||
off_below: 0.0
|
|
||||||
heater: extruder
|
|
||||||
fan_speed: 1.0
|
|
||||||
heater_temp: 60.0
|
|
||||||
|
|
||||||
|
|
||||||
[bed_mesh]
|
|
||||||
speed: 350
|
|
||||||
mesh_min: 5,10 #need to handle head distance with bl_touch
|
|
||||||
mesh_max: 215,215 #max probe range
|
|
||||||
probe_count: 5,5
|
|
||||||
fade_start: 1
|
|
||||||
fade_end: 10
|
|
||||||
fade_target: 0
|
|
||||||
horizontal_move_z: 8
|
|
||||||
|
|
||||||
[input_shaper]
|
|
||||||
shaper_type_y = mzv
|
|
||||||
shaper_freq_y = 32.0
|
|
||||||
|
|
||||||
shaper_type_x = mzv
|
|
||||||
shaper_freq_x = 38.5
|
|
||||||
|
|
||||||
[mcu rpi]
|
|
||||||
serial: /tmp/klipper_host_mcu
|
|
||||||
|
|
||||||
[adxl345]
|
|
||||||
cs_pin: rpi:None
|
|
||||||
spi_speed: 2000000
|
|
||||||
spi_bus: spidev2.0
|
|
||||||
axes_map: z,y,x
|
|
||||||
|
|
||||||
[resonance_tester]
|
|
||||||
accel_chip: adxl345
|
|
||||||
accel_per_hz: 70
|
|
||||||
probe_points: 117.5,117.5,100
|
|
||||||
max_freq: 90
|
|
||||||
@@ -1,705 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""
|
|
||||||
Ender-3 V3 KE firmware extractor/repacker.
|
|
||||||
|
|
||||||
This script handles the stock Creality 7z-wrapped `.img` package, reconstructs
|
|
||||||
the split payload files, and can rebuild a fresh package from an extracted
|
|
||||||
rootfs tree.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import hashlib
|
|
||||||
import os
|
|
||||||
import shlex
|
|
||||||
import re
|
|
||||||
import shutil
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import tempfile
|
|
||||||
import urllib.request
|
|
||||||
import zipfile
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import List, Sequence, Tuple
|
|
||||||
|
|
||||||
import py7zr
|
|
||||||
|
|
||||||
|
|
||||||
PASSWORD = "$1$cxswfile$ZFd0RWFYkJQugbtKVGL9y0"
|
|
||||||
CHUNK_SIZE = 1024 * 1024
|
|
||||||
UIMAGE_HEADER_SIZE = 64
|
|
||||||
DEFAULT_STOCK_ARCHIVE = Path("Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img")
|
|
||||||
DEFAULT_OVERLAY_DIR = Path("overlay_rootfs")
|
|
||||||
DEFAULT_VERSION = "11034"
|
|
||||||
SQUASHFS_TOOLS_URL = "https://infraroot.at/pub/squashfs/windows/squashfs-tools-ng-1.3.2-mingw64.zip"
|
|
||||||
SQUASHFS_TOOLS_DIR = Path("build") / "_toolcache" / "squashfs-tools-ng-1.3.2-mingw64"
|
|
||||||
|
|
||||||
|
|
||||||
def md5_file(path: Path) -> str:
|
|
||||||
digest = hashlib.md5()
|
|
||||||
with path.open("rb") as handle:
|
|
||||||
for chunk in iter(lambda: handle.read(4 * 1024 * 1024), b""):
|
|
||||||
digest.update(chunk)
|
|
||||||
return digest.hexdigest()
|
|
||||||
|
|
||||||
|
|
||||||
def ensure_clean_dir(path: Path) -> None:
|
|
||||||
if path.exists():
|
|
||||||
shutil.rmtree(path)
|
|
||||||
path.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
|
|
||||||
def reconstruct_split_file(src_dir: Path, prefix: str, out_path: Path) -> List[Path]:
|
|
||||||
pattern = re.compile(rf"^{re.escape(prefix)}\.(\d{{4}})\.[0-9a-f]{{32}}$")
|
|
||||||
parts: List[Tuple[int, Path]] = []
|
|
||||||
for entry in src_dir.iterdir():
|
|
||||||
if not entry.is_file():
|
|
||||||
continue
|
|
||||||
match = pattern.match(entry.name)
|
|
||||||
if match:
|
|
||||||
parts.append((int(match.group(1)), entry))
|
|
||||||
if not parts:
|
|
||||||
raise FileNotFoundError(f"no split files found for {prefix} in {src_dir}")
|
|
||||||
|
|
||||||
parts.sort(key=lambda item: item[0])
|
|
||||||
with out_path.open("wb") as out_handle:
|
|
||||||
for _, part in parts:
|
|
||||||
out_handle.write(part.read_bytes())
|
|
||||||
return [path for _, path in parts]
|
|
||||||
|
|
||||||
|
|
||||||
def write_lines(path: Path, lines: Sequence[str]) -> None:
|
|
||||||
path.write_text("\n".join(lines) + "\n", encoding="utf-8", newline="\n")
|
|
||||||
|
|
||||||
|
|
||||||
def split_file(src_path: Path, dest_dir: Path, base_name: str) -> Tuple[str, List[Path], List[str]]:
|
|
||||||
full_md5 = md5_file(src_path)
|
|
||||||
chunk_md5s: List[str] = []
|
|
||||||
chunk_paths: List[Path] = []
|
|
||||||
with src_path.open("rb") as handle:
|
|
||||||
index = 0
|
|
||||||
previous_md5 = full_md5
|
|
||||||
while True:
|
|
||||||
data = handle.read(CHUNK_SIZE)
|
|
||||||
if not data:
|
|
||||||
break
|
|
||||||
chunk_md5 = hashlib.md5(data).hexdigest()
|
|
||||||
chunk_name = f"{base_name}.{index:04d}.{previous_md5}"
|
|
||||||
chunk_path = dest_dir / chunk_name
|
|
||||||
chunk_path.write_bytes(data)
|
|
||||||
chunk_paths.append(chunk_path)
|
|
||||||
chunk_md5s.append(chunk_md5)
|
|
||||||
previous_md5 = chunk_md5
|
|
||||||
index += 1
|
|
||||||
return full_md5, chunk_paths, chunk_md5s
|
|
||||||
|
|
||||||
|
|
||||||
def download_file(url: str, dest: Path) -> None:
|
|
||||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
temp_dest = dest.with_name(dest.name + ".download")
|
|
||||||
if temp_dest.exists():
|
|
||||||
temp_dest.unlink()
|
|
||||||
with urllib.request.urlopen(url) as response, temp_dest.open("wb") as out_handle:
|
|
||||||
shutil.copyfileobj(response, out_handle)
|
|
||||||
temp_dest.replace(dest)
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_squashfs_tool(tool_name: str) -> Path:
|
|
||||||
"""
|
|
||||||
Resolve a squashfs-tools-ng executable for the current host.
|
|
||||||
|
|
||||||
On Windows we use the cached MinGW bundle that ships with this repo.
|
|
||||||
On other platforms we require a native binary from PATH.
|
|
||||||
"""
|
|
||||||
|
|
||||||
if os.name == "nt":
|
|
||||||
tools_bin = ensure_squashfs_tools(SQUASHFS_TOOLS_DIR, SQUASHFS_TOOLS_URL)
|
|
||||||
return tools_bin / f"{tool_name}.exe"
|
|
||||||
|
|
||||||
resolved = shutil.which(tool_name)
|
|
||||||
if resolved:
|
|
||||||
return Path(resolved)
|
|
||||||
|
|
||||||
raise FileNotFoundError(
|
|
||||||
f"{tool_name} not found on PATH. Install squashfs-tools-ng for your Linux distribution "
|
|
||||||
f"or run the helper on Windows where the bundled MinGW tools are available."
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def extract_archive(archive: Path, outdir: Path, password: str) -> Path:
|
|
||||||
ensure_clean_dir(outdir)
|
|
||||||
with py7zr.SevenZipFile(archive, mode="r", password=password) as zf:
|
|
||||||
zf.extractall(path=outdir)
|
|
||||||
|
|
||||||
top_entries = [entry for entry in outdir.iterdir() if entry.is_dir()]
|
|
||||||
if len(top_entries) != 1:
|
|
||||||
raise RuntimeError(f"expected one top-level folder after extraction, found {len(top_entries)}")
|
|
||||||
return top_entries[0]
|
|
||||||
|
|
||||||
|
|
||||||
def ensure_stock_archive(stock_archive: Path, stock_url: str | None, password: str) -> Path:
|
|
||||||
if stock_archive.exists():
|
|
||||||
return stock_archive
|
|
||||||
if not stock_url:
|
|
||||||
raise FileNotFoundError(
|
|
||||||
f"stock archive not found at {stock_archive}; supply --stock-url to download it first"
|
|
||||||
)
|
|
||||||
download_file(stock_url, stock_archive)
|
|
||||||
with py7zr.SevenZipFile(stock_archive, mode="r", password=password) as zf:
|
|
||||||
zf.test()
|
|
||||||
return stock_archive
|
|
||||||
|
|
||||||
|
|
||||||
def ensure_squashfs_tools(tools_dir: Path, tools_url: str) -> Path:
|
|
||||||
bin_dir = tools_dir / "bin"
|
|
||||||
gensquashfs = bin_dir / "gensquashfs.exe"
|
|
||||||
rdsquashfs = bin_dir / "rdsquashfs.exe"
|
|
||||||
if gensquashfs.exists() and rdsquashfs.exists():
|
|
||||||
return bin_dir
|
|
||||||
|
|
||||||
tools_dir.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
zip_path = tools_dir.with_suffix(".zip")
|
|
||||||
if not zip_path.exists():
|
|
||||||
download_file(tools_url, zip_path)
|
|
||||||
if tools_dir.exists():
|
|
||||||
shutil.rmtree(tools_dir)
|
|
||||||
with zipfile.ZipFile(zip_path) as zf:
|
|
||||||
zf.extractall(path=tools_dir.parent)
|
|
||||||
|
|
||||||
if not gensquashfs.exists() or not rdsquashfs.exists():
|
|
||||||
raise FileNotFoundError(f"failed to install squashfs tools into {bin_dir}")
|
|
||||||
return bin_dir
|
|
||||||
|
|
||||||
|
|
||||||
def reconstruct_from_archive_tree(tree_root: Path, workspace: Path) -> dict[str, Path]:
|
|
||||||
version_file = tree_root / "ota_config.in"
|
|
||||||
if version_file.exists():
|
|
||||||
version = read_version(version_file)
|
|
||||||
ota_dir = tree_root / f"ota_v{version}"
|
|
||||||
else:
|
|
||||||
ota_dir = tree_root / f"ota_v{DEFAULT_VERSION}"
|
|
||||||
|
|
||||||
if not ota_dir.exists():
|
|
||||||
matches = sorted([entry for entry in tree_root.iterdir() if entry.is_dir() and entry.name.startswith("ota_v")])
|
|
||||||
if not matches:
|
|
||||||
raise FileNotFoundError(f"could not locate ota_v* directory under {tree_root}")
|
|
||||||
ota_dir = matches[0]
|
|
||||||
|
|
||||||
rootfs_full = workspace / "rootfs.squashfs.full"
|
|
||||||
ximage_full = workspace / "xImage.full"
|
|
||||||
zero_full = workspace / "zero.bin.full"
|
|
||||||
rootfs_full.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
ximage_full.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
zero_full.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
reconstruct_split_file(ota_dir, "rootfs.squashfs", rootfs_full)
|
|
||||||
reconstruct_split_file(ota_dir, "xImage", ximage_full)
|
|
||||||
reconstruct_split_file(ota_dir, "zero.bin", zero_full)
|
|
||||||
|
|
||||||
ximage_payload = workspace / "xImage.payload"
|
|
||||||
ximage_payload.write_bytes(ximage_full.read_bytes()[UIMAGE_HEADER_SIZE:])
|
|
||||||
|
|
||||||
return {
|
|
||||||
"ota_dir": ota_dir,
|
|
||||||
"rootfs_full": rootfs_full,
|
|
||||||
"ximage_full": ximage_full,
|
|
||||||
"ximage_payload": ximage_payload,
|
|
||||||
"zero_full": zero_full,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def extract_rootfs_to_dir(rootfs_full: Path, outdir: Path) -> None:
|
|
||||||
ensure_clean_dir(outdir)
|
|
||||||
rdsquashfs = resolve_squashfs_tool("rdsquashfs")
|
|
||||||
subprocess.run(
|
|
||||||
[
|
|
||||||
str(rdsquashfs),
|
|
||||||
"--unpack-path",
|
|
||||||
"/",
|
|
||||||
"--unpack-root",
|
|
||||||
str(outdir),
|
|
||||||
"--chmod",
|
|
||||||
"--chown",
|
|
||||||
"--set-times",
|
|
||||||
"--quiet",
|
|
||||||
str(rootfs_full),
|
|
||||||
],
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def describe_rootfs(rootfs_full: Path) -> List[str]:
|
|
||||||
rdsquashfs = resolve_squashfs_tool("rdsquashfs")
|
|
||||||
result = subprocess.run(
|
|
||||||
[str(rdsquashfs), "--describe", str(rootfs_full)],
|
|
||||||
check=True,
|
|
||||||
capture_output=True,
|
|
||||||
text=True,
|
|
||||||
encoding="utf-8",
|
|
||||||
errors="replace",
|
|
||||||
)
|
|
||||||
return [line.strip() for line in result.stdout.splitlines() if line.strip()]
|
|
||||||
|
|
||||||
|
|
||||||
def describe_rootfs_map(rootfs_full: Path) -> dict[str, str]:
|
|
||||||
root_entries: dict[str, str] = {}
|
|
||||||
for line in describe_rootfs(rootfs_full):
|
|
||||||
parts = shlex.split(line)
|
|
||||||
if len(parts) < 2:
|
|
||||||
continue
|
|
||||||
root_entries[parts[1].lstrip("/")] = line
|
|
||||||
return root_entries
|
|
||||||
|
|
||||||
|
|
||||||
def packfile_entry(line: str) -> tuple[str, str]:
|
|
||||||
parts = shlex.split(line)
|
|
||||||
if len(parts) < 5:
|
|
||||||
raise ValueError(f"invalid squashfs describe line: {line}")
|
|
||||||
kind = parts[0]
|
|
||||||
path = parts[1]
|
|
||||||
mode = parts[2]
|
|
||||||
uid = parts[3]
|
|
||||||
gid = parts[4]
|
|
||||||
extra = " ".join(parts[5:])
|
|
||||||
pack_path = path if path.startswith("/") else f"/{path}"
|
|
||||||
if kind in {"dir", "file"}:
|
|
||||||
entry = f"{kind} {quote_pack_path(pack_path)} {mode} {uid} {gid}"
|
|
||||||
elif kind in {"slink", "link"}:
|
|
||||||
if not extra:
|
|
||||||
raise ValueError(f"missing link target for {line}")
|
|
||||||
entry = f"{kind} {quote_pack_path(pack_path)} {mode} {uid} {gid} {extra}"
|
|
||||||
elif kind == "nod":
|
|
||||||
if not extra:
|
|
||||||
raise ValueError(f"missing device metadata for {line}")
|
|
||||||
entry = f"{kind} {quote_pack_path(pack_path)} {mode} {uid} {gid} {extra}"
|
|
||||||
else:
|
|
||||||
raise ValueError(f"unsupported squashfs entry type: {kind}")
|
|
||||||
return kind, entry
|
|
||||||
|
|
||||||
|
|
||||||
def quote_pack_path(path: str) -> str:
|
|
||||||
if not any(ch.isspace() for ch in path) and '"' not in path:
|
|
||||||
return path
|
|
||||||
return '"' + path.replace("\\", "\\\\").replace('"', '\\"') + '"'
|
|
||||||
|
|
||||||
|
|
||||||
def list_overlay_entries(rootfs_dir: Path, stock_paths: set[str]) -> List[str]:
|
|
||||||
additions: List[tuple[int, str]] = []
|
|
||||||
for entry in rootfs_dir.rglob("*"):
|
|
||||||
relative = entry.relative_to(rootfs_dir).as_posix()
|
|
||||||
pack_path = f"/{relative}"
|
|
||||||
if relative in stock_paths:
|
|
||||||
continue
|
|
||||||
if entry.is_dir() and not entry.is_symlink():
|
|
||||||
additions.append((0, f"dir {quote_pack_path(pack_path)} 0755 0 0"))
|
|
||||||
elif entry.is_symlink():
|
|
||||||
additions.append((2, f"slink {quote_pack_path(pack_path)} 0777 0 0 {os.readlink(entry)}"))
|
|
||||||
else:
|
|
||||||
mode_bits = entry.stat().st_mode & 0o7777
|
|
||||||
if mode_bits & 0o111 == 0 and relative.startswith("etc/init.d/"):
|
|
||||||
mode_bits = (mode_bits & ~0o666) | 0o755
|
|
||||||
mode = oct(mode_bits)[2:]
|
|
||||||
additions.append((1, f"file {quote_pack_path(pack_path)} {mode} 0 0"))
|
|
||||||
additions.sort(key=lambda item: (item[0], item[1]))
|
|
||||||
return [entry for _, entry in additions]
|
|
||||||
|
|
||||||
|
|
||||||
def collect_intentional_overlay_paths(overlay_dir: Path) -> set[str]:
|
|
||||||
intentional: set[str] = set()
|
|
||||||
delete_file = overlay_dir / ".delete"
|
|
||||||
if delete_file.exists():
|
|
||||||
intentional.update(
|
|
||||||
str(path).replace("\\", "/")
|
|
||||||
for path in read_delete_list(overlay_dir)
|
|
||||||
)
|
|
||||||
|
|
||||||
for entry in overlay_dir.rglob("*"):
|
|
||||||
relative = entry.relative_to(overlay_dir).as_posix()
|
|
||||||
intentional.add(relative)
|
|
||||||
return intentional
|
|
||||||
|
|
||||||
|
|
||||||
def should_skip_compare_path(path: str) -> bool:
|
|
||||||
return any(ord(ch) > 127 for ch in path)
|
|
||||||
|
|
||||||
|
|
||||||
def compare_rootfs_metadata(stock_rootfs_full: Path, rebuilt_rootfs_full: Path, overlay_dir: Path) -> List[str]:
|
|
||||||
stock_map = describe_rootfs_map(stock_rootfs_full)
|
|
||||||
rebuilt_map = describe_rootfs_map(rebuilt_rootfs_full)
|
|
||||||
intentional = collect_intentional_overlay_paths(overlay_dir)
|
|
||||||
|
|
||||||
issues: List[str] = []
|
|
||||||
for path, stock_line in stock_map.items():
|
|
||||||
if should_skip_compare_path(path):
|
|
||||||
continue
|
|
||||||
if path in intentional:
|
|
||||||
continue
|
|
||||||
rebuilt_line = rebuilt_map.get(path)
|
|
||||||
if rebuilt_line is None:
|
|
||||||
issues.append(f"missing from rebuilt rootfs: {path}")
|
|
||||||
continue
|
|
||||||
if rebuilt_line != stock_line:
|
|
||||||
issues.append(f"metadata mismatch: {path}")
|
|
||||||
|
|
||||||
for path, rebuilt_line in rebuilt_map.items():
|
|
||||||
if should_skip_compare_path(path):
|
|
||||||
continue
|
|
||||||
if path in stock_map or path in intentional:
|
|
||||||
continue
|
|
||||||
issues.append(f"unexpected extra path in rebuilt rootfs: {path}")
|
|
||||||
|
|
||||||
return issues
|
|
||||||
|
|
||||||
|
|
||||||
def write_pack_file(rootfs_dir: Path, stock_rootfs_full: Path, packfile_path: Path) -> None:
|
|
||||||
lines = describe_rootfs(stock_rootfs_full)
|
|
||||||
stock_entries: List[str] = []
|
|
||||||
stock_paths: set[str] = set()
|
|
||||||
for line in lines:
|
|
||||||
kind, entry = packfile_entry(line)
|
|
||||||
path = shlex.split(line)[1]
|
|
||||||
stock_paths.add(path)
|
|
||||||
target_path = rootfs_dir / path
|
|
||||||
if kind in {"file", "dir"}:
|
|
||||||
if not target_path.exists():
|
|
||||||
continue
|
|
||||||
elif kind in {"slink", "link", "nod"}:
|
|
||||||
if kind in {"slink", "link"} and target_path.exists():
|
|
||||||
remove_path(target_path)
|
|
||||||
stock_entries.append(entry)
|
|
||||||
|
|
||||||
overlay_entries = list_overlay_entries(rootfs_dir, stock_paths)
|
|
||||||
packfile_path.write_text("\n".join(stock_entries + overlay_entries) + "\n", encoding="utf-8", newline="\n")
|
|
||||||
|
|
||||||
|
|
||||||
def remove_path(path: Path) -> None:
|
|
||||||
if not path.exists() and not path.is_symlink():
|
|
||||||
return
|
|
||||||
if path.is_dir() and not path.is_symlink():
|
|
||||||
shutil.rmtree(path)
|
|
||||||
else:
|
|
||||||
path.unlink()
|
|
||||||
|
|
||||||
|
|
||||||
def safe_rmtree(path: Path) -> None:
|
|
||||||
if not path.exists() and not path.is_symlink():
|
|
||||||
return
|
|
||||||
if path.is_symlink() or path.is_file():
|
|
||||||
path.unlink(missing_ok=True)
|
|
||||||
return
|
|
||||||
for entry in path.iterdir():
|
|
||||||
safe_rmtree(entry)
|
|
||||||
path.rmdir()
|
|
||||||
|
|
||||||
|
|
||||||
def read_delete_list(overlay_dir: Path) -> List[Path]:
|
|
||||||
delete_file = overlay_dir / ".delete"
|
|
||||||
if not delete_file.exists():
|
|
||||||
return []
|
|
||||||
|
|
||||||
deletions: List[Path] = []
|
|
||||||
for raw_line in delete_file.read_text(encoding="utf-8").splitlines():
|
|
||||||
line = raw_line.strip()
|
|
||||||
if not line or line.startswith("#"):
|
|
||||||
continue
|
|
||||||
deletions.append(Path(line))
|
|
||||||
return deletions
|
|
||||||
|
|
||||||
|
|
||||||
def apply_overlay_tree(rootfs_dir: Path, overlay_dir: Path) -> None:
|
|
||||||
if not overlay_dir.exists():
|
|
||||||
raise FileNotFoundError(f"overlay directory not found: {overlay_dir}")
|
|
||||||
|
|
||||||
for relative in read_delete_list(overlay_dir):
|
|
||||||
target = rootfs_dir / relative
|
|
||||||
remove_path(target)
|
|
||||||
|
|
||||||
for entry in overlay_dir.rglob("*"):
|
|
||||||
if entry.is_dir():
|
|
||||||
continue
|
|
||||||
if entry.name == ".delete" and entry.parent == overlay_dir:
|
|
||||||
continue
|
|
||||||
|
|
||||||
relative = entry.relative_to(overlay_dir)
|
|
||||||
target = rootfs_dir / relative
|
|
||||||
target.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
remove_path(target)
|
|
||||||
shutil.copy2(entry, target)
|
|
||||||
|
|
||||||
|
|
||||||
def build_rootfs_from_dir(rootfs_dir: Path, stock_rootfs_full: Path, output_path: Path, workspace: Path) -> None:
|
|
||||||
gensquashfs = resolve_squashfs_tool("gensquashfs")
|
|
||||||
packfile_path = workspace / "rootfs.packfile"
|
|
||||||
write_pack_file(rootfs_dir, stock_rootfs_full, packfile_path)
|
|
||||||
if output_path.exists():
|
|
||||||
output_path.unlink()
|
|
||||||
subprocess.run(
|
|
||||||
[
|
|
||||||
str(gensquashfs),
|
|
||||||
"--pack-file",
|
|
||||||
str(packfile_path),
|
|
||||||
"--pack-dir",
|
|
||||||
str(rootfs_dir),
|
|
||||||
"--keep-time",
|
|
||||||
"--block-size",
|
|
||||||
"131072",
|
|
||||||
"--compressor",
|
|
||||||
"gzip",
|
|
||||||
"--force",
|
|
||||||
"--quiet",
|
|
||||||
str(output_path),
|
|
||||||
],
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def read_version(config_path: Path) -> str:
|
|
||||||
for line in config_path.read_text(encoding="utf-8").splitlines():
|
|
||||||
line = line.strip()
|
|
||||||
if line.startswith("current_version="):
|
|
||||||
return line.split("=", 1)[1].strip()
|
|
||||||
raise ValueError(f"current_version not found in {config_path}")
|
|
||||||
|
|
||||||
|
|
||||||
def write_ota_manifest(
|
|
||||||
staging_root: Path,
|
|
||||||
version: str,
|
|
||||||
items: Sequence[Tuple[str, str, Path, Path, List[str]]],
|
|
||||||
) -> None:
|
|
||||||
archive_root = staging_root.name
|
|
||||||
version_dir = staging_root / f"ota_v{version}"
|
|
||||||
version_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
(staging_root / "ota_config.in").write_text(f"current_version={version}\n", encoding="utf-8")
|
|
||||||
(version_dir / f"ota_v{version}.ok").write_bytes(b"")
|
|
||||||
|
|
||||||
ota_update_lines = [f"ota_version={version}", ""]
|
|
||||||
for img_type, img_name, full_path, md5_path, chunk_md5s in items:
|
|
||||||
ota_update_lines.extend(
|
|
||||||
[
|
|
||||||
f"img_type={img_type}",
|
|
||||||
f"img_name={img_name}",
|
|
||||||
f"img_size={full_path.stat().st_size}",
|
|
||||||
f"img_md5={md5_file(full_path)}",
|
|
||||||
"",
|
|
||||||
]
|
|
||||||
)
|
|
||||||
sidecar_name = f"ota_md5_{img_name}.{md5_file(full_path)}"
|
|
||||||
write_lines(version_dir / sidecar_name, chunk_md5s)
|
|
||||||
|
|
||||||
write_lines(version_dir / "ota_update.in", ota_update_lines)
|
|
||||||
|
|
||||||
|
|
||||||
def stage_and_pack(
|
|
||||||
workspace: Path,
|
|
||||||
version: str,
|
|
||||||
rootfs_full: Path,
|
|
||||||
ximage_full: Path,
|
|
||||||
zero_full: Path,
|
|
||||||
output_img: Path,
|
|
||||||
password: str,
|
|
||||||
) -> Path:
|
|
||||||
archive_root = f"DarKE_KE_F005_ota_img_V{version}"
|
|
||||||
staging_root = workspace / "_firmware_stage" / archive_root
|
|
||||||
ensure_clean_dir(staging_root.parent)
|
|
||||||
staging_root.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
version_dir = staging_root / f"ota_v{version}"
|
|
||||||
version_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
rootfs_md5, rootfs_chunks, rootfs_chunk_md5s = split_file(rootfs_full, version_dir, "rootfs.squashfs")
|
|
||||||
ximage_md5, ximage_chunks, ximage_chunk_md5s = split_file(ximage_full, version_dir, "xImage")
|
|
||||||
zero_md5, zero_chunks, zero_chunk_md5s = split_file(zero_full, version_dir, "zero.bin")
|
|
||||||
|
|
||||||
(staging_root / "ota_config.in").write_text(f"current_version={version}\n", encoding="utf-8", newline="\n")
|
|
||||||
(version_dir / f"ota_v{version}.ok").write_bytes(b"")
|
|
||||||
|
|
||||||
ota_update_lines = [f"ota_version={version}", ""]
|
|
||||||
for img_type, img_name, full_path, chunk_md5s, full_md5 in [
|
|
||||||
("kernel", "xImage", ximage_full, ximage_chunk_md5s, ximage_md5),
|
|
||||||
("rootfs", "rootfs.squashfs", rootfs_full, rootfs_chunk_md5s, rootfs_md5),
|
|
||||||
("rtos", "zero.bin", zero_full, zero_chunk_md5s, zero_md5),
|
|
||||||
]:
|
|
||||||
ota_update_lines.extend(
|
|
||||||
[
|
|
||||||
f"img_type={img_type}",
|
|
||||||
f"img_name={img_name}",
|
|
||||||
f"img_size={full_path.stat().st_size}",
|
|
||||||
f"img_md5={full_md5}",
|
|
||||||
"",
|
|
||||||
]
|
|
||||||
)
|
|
||||||
sidecar_name = f"ota_md5_{img_name}.{full_md5}"
|
|
||||||
write_lines(version_dir / sidecar_name, chunk_md5s)
|
|
||||||
|
|
||||||
write_lines(version_dir / "ota_update.in", ota_update_lines)
|
|
||||||
|
|
||||||
if output_img.exists():
|
|
||||||
output_img.unlink()
|
|
||||||
with py7zr.SevenZipFile(output_img, mode="w", password=password) as zf:
|
|
||||||
zf.writeall(staging_root, arcname=archive_root)
|
|
||||||
|
|
||||||
return output_img
|
|
||||||
|
|
||||||
|
|
||||||
def validate_archive(archive: Path, password: str) -> None:
|
|
||||||
with tempfile.TemporaryDirectory(dir=str(archive.parent)) as tmpdir:
|
|
||||||
tmp_path = Path(tmpdir)
|
|
||||||
with py7zr.SevenZipFile(archive, mode="r", password=password) as zf:
|
|
||||||
zf.extractall(path=tmp_path)
|
|
||||||
top = next(tmp_path.iterdir(), None)
|
|
||||||
if top is None or not top.is_dir():
|
|
||||||
raise RuntimeError("archive validation failed: missing top-level directory")
|
|
||||||
ota_config = top / "ota_config.in"
|
|
||||||
if not ota_config.exists():
|
|
||||||
raise RuntimeError("archive validation failed: missing ota_config.in")
|
|
||||||
|
|
||||||
|
|
||||||
def cmd_extract(args: argparse.Namespace) -> None:
|
|
||||||
archive = Path(args.archive).expanduser().resolve()
|
|
||||||
outdir = Path(args.outdir).expanduser().resolve()
|
|
||||||
workspace = Path(args.workspace).expanduser().resolve()
|
|
||||||
tree_root = extract_archive(archive, outdir, args.password)
|
|
||||||
files = reconstruct_from_archive_tree(tree_root, workspace)
|
|
||||||
extract_rootfs_to_dir(files["rootfs_full"], workspace / "rootfs_extract")
|
|
||||||
print(f"extracted archive to {outdir}")
|
|
||||||
print(f"reconstructed rootfs: {files['rootfs_full']}")
|
|
||||||
print(f"reconstructed kernel: {files['ximage_full']}")
|
|
||||||
print(f"reconstructed rtos: {files['zero_full']}")
|
|
||||||
print(f"rootfs unpacked to: {workspace / 'rootfs_extract'}")
|
|
||||||
|
|
||||||
|
|
||||||
def cmd_build(args: argparse.Namespace) -> None:
|
|
||||||
version = args.version
|
|
||||||
if not version:
|
|
||||||
version = DEFAULT_VERSION
|
|
||||||
|
|
||||||
build_dir = Path(args.build_dir).expanduser().resolve()
|
|
||||||
build_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
output_img = Path(args.output).expanduser().resolve()
|
|
||||||
stock_archive = ensure_stock_archive(
|
|
||||||
Path(args.stock_archive).expanduser().resolve(),
|
|
||||||
args.stock_url,
|
|
||||||
args.password,
|
|
||||||
)
|
|
||||||
overlay_dir = Path(args.overlay_dir).expanduser().resolve()
|
|
||||||
|
|
||||||
workdir = Path(tempfile.mkdtemp(prefix="_firmware_build_", dir=str(build_dir)))
|
|
||||||
try:
|
|
||||||
stock_extract_dir = workdir / "stock_extract"
|
|
||||||
stock_tree = extract_archive(stock_archive, stock_extract_dir, args.password)
|
|
||||||
stock_files = reconstruct_from_archive_tree(stock_tree, workdir)
|
|
||||||
|
|
||||||
rootfs_dir = workdir / "rootfs_work"
|
|
||||||
extract_rootfs_to_dir(stock_files["rootfs_full"], rootfs_dir)
|
|
||||||
apply_overlay_tree(rootfs_dir, overlay_dir)
|
|
||||||
|
|
||||||
rootfs_full = workdir / "rootfs.squashfs.full"
|
|
||||||
build_rootfs_from_dir(rootfs_dir, stock_files["rootfs_full"], rootfs_full, workdir)
|
|
||||||
stage_and_pack(
|
|
||||||
workspace=workdir,
|
|
||||||
version=version,
|
|
||||||
rootfs_full=rootfs_full,
|
|
||||||
ximage_full=stock_files["ximage_full"],
|
|
||||||
zero_full=stock_files["zero_full"],
|
|
||||||
output_img=output_img,
|
|
||||||
password=args.password,
|
|
||||||
)
|
|
||||||
validate_archive(output_img, args.password)
|
|
||||||
finally:
|
|
||||||
safe_rmtree(workdir)
|
|
||||||
|
|
||||||
print(f"built firmware image: {output_img}")
|
|
||||||
|
|
||||||
|
|
||||||
def cmd_compare(args: argparse.Namespace) -> None:
|
|
||||||
stock_archive = Path(args.stock_archive).expanduser().resolve()
|
|
||||||
rebuilt_archive = Path(args.rebuilt_archive).expanduser().resolve()
|
|
||||||
overlay_dir = Path(args.overlay_dir).expanduser().resolve()
|
|
||||||
compare_root = Path(args.work_dir).expanduser().resolve()
|
|
||||||
compare_root.mkdir(parents=True, exist_ok=True)
|
|
||||||
workdir = Path(tempfile.mkdtemp(prefix="_firmware_compare_", dir=str(compare_root)))
|
|
||||||
try:
|
|
||||||
stock_tree = extract_archive(stock_archive, workdir / "stock_extract", args.password)
|
|
||||||
rebuilt_tree = extract_archive(rebuilt_archive, workdir / "rebuilt_extract", args.password)
|
|
||||||
stock_files = reconstruct_from_archive_tree(stock_tree, workdir / "stock")
|
|
||||||
rebuilt_files = reconstruct_from_archive_tree(rebuilt_tree, workdir / "rebuilt")
|
|
||||||
stock_rootfs_full = stock_files["rootfs_full"]
|
|
||||||
rebuilt_rootfs_full = rebuilt_files["rootfs_full"]
|
|
||||||
issues = compare_rootfs_metadata(stock_rootfs_full, rebuilt_rootfs_full, overlay_dir)
|
|
||||||
if issues:
|
|
||||||
for issue in issues:
|
|
||||||
print(issue)
|
|
||||||
raise SystemExit(1)
|
|
||||||
finally:
|
|
||||||
safe_rmtree(workdir)
|
|
||||||
|
|
||||||
print("rootfs metadata matches stock outside the overlay")
|
|
||||||
|
|
||||||
|
|
||||||
def build_parser() -> argparse.ArgumentParser:
|
|
||||||
parser = argparse.ArgumentParser(description="Extract and rebuild Ender-3 V3 KE firmware images.")
|
|
||||||
parser.add_argument("--password", default=PASSWORD, help="7z archive password")
|
|
||||||
subparsers = parser.add_subparsers(dest="command", required=True)
|
|
||||||
|
|
||||||
extract = subparsers.add_parser("extract", help="extract an official firmware archive")
|
|
||||||
extract.add_argument("--archive", required=True, help="path to the stock .img archive")
|
|
||||||
extract.add_argument("--outdir", default="ke_firmware_extract", help="archive extraction directory")
|
|
||||||
extract.add_argument("--workspace", default=".", help="workspace for reconstructed artifacts")
|
|
||||||
extract.set_defaults(func=cmd_extract)
|
|
||||||
|
|
||||||
build = subparsers.add_parser("build", help="build a repacked firmware archive")
|
|
||||||
build.add_argument("--workspace", default=".", help="workspace root")
|
|
||||||
build.add_argument("--build-dir", default="build", help="directory for intermediate build artifacts")
|
|
||||||
build.add_argument(
|
|
||||||
"--stock-archive",
|
|
||||||
default=str(DEFAULT_STOCK_ARCHIVE),
|
|
||||||
help="path to the stock OTA image to use as the build base",
|
|
||||||
)
|
|
||||||
build.add_argument(
|
|
||||||
"--stock-url",
|
|
||||||
default=None,
|
|
||||||
help="download URL used when the stock OTA image is missing",
|
|
||||||
)
|
|
||||||
build.add_argument(
|
|
||||||
"--overlay-dir",
|
|
||||||
default=str(DEFAULT_OVERLAY_DIR),
|
|
||||||
help="directory containing only our custom rootfs changes",
|
|
||||||
)
|
|
||||||
build.add_argument(
|
|
||||||
"--output",
|
|
||||||
default=str(Path("build") / f"DarKE_KE_F005_ota_img_V{DEFAULT_VERSION}.img"),
|
|
||||||
help="output .img archive path",
|
|
||||||
)
|
|
||||||
build.add_argument("--version", default=DEFAULT_VERSION, help="firmware version to encode in manifests")
|
|
||||||
build.set_defaults(func=cmd_build)
|
|
||||||
|
|
||||||
compare = subparsers.add_parser("compare", help="compare a rebuilt image's rootfs metadata against stock")
|
|
||||||
compare.add_argument("--stock-archive", required=True, help="path to the stock .img archive")
|
|
||||||
compare.add_argument("--rebuilt-archive", required=True, help="path to the rebuilt .img archive")
|
|
||||||
compare.add_argument(
|
|
||||||
"--overlay-dir",
|
|
||||||
default=str(DEFAULT_OVERLAY_DIR),
|
|
||||||
help="overlay tree used to produce the rebuilt image",
|
|
||||||
)
|
|
||||||
compare.add_argument("--work-dir", default="build", help="directory for temporary compare artifacts")
|
|
||||||
compare.set_defaults(func=cmd_compare)
|
|
||||||
|
|
||||||
return parser
|
|
||||||
|
|
||||||
|
|
||||||
def main(argv: Sequence[str] | None = None) -> int:
|
|
||||||
for stream in (sys.stdout, sys.stderr):
|
|
||||||
try:
|
|
||||||
stream.reconfigure(encoding="utf-8", errors="replace")
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
parser = build_parser()
|
|
||||||
args = parser.parse_args(argv)
|
|
||||||
args.func(args)
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
Executable
+505
@@ -0,0 +1,505 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
OpenKE Firmware Packager for Ender-3 V3 KE and Creality Nebula Pad.
|
||||||
|
|
||||||
|
Packages OpenKE kernel (xImage) and rootfs (rootfs.squashfs) along with the board-specific
|
||||||
|
RTOS firmware (zero.bin) into a stock CrealityOS-supported encrypted 7z OTA (.img) update.
|
||||||
|
|
||||||
|
CRITICAL REQUIREMENT (from /etc/ota_bin/local_ota_update.sh):
|
||||||
|
OTA_UNZIP_FILE_NAME=${OTA_FILE_NAME%.img*}
|
||||||
|
ota_site=${OTA_FILE_PATH}/${OTA_UNZIP_FILE_NAME}
|
||||||
|
ota_site_config=$ota_site/ota_config.in
|
||||||
|
The top-level folder inside the 7z archive MUST EXACTLY MATCH the .img filename (minus .img),
|
||||||
|
and the version must match Creality's dotted versioning (e.g. 1.1.0.34 > 1.1.0.30).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import hashlib
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Dict, List, Sequence, Tuple
|
||||||
|
|
||||||
|
# Fixed hardware partition limits for X2000 Creality display boards
|
||||||
|
KERNEL_PART_MAX_BYTES = 8 * 1024 * 1024 # 8 MiB (mmcblk0p5 / mmcblk0p6)
|
||||||
|
ROOTFS_PART_MAX_BYTES = 500 * 1024 * 1024 # 500 MiB (mmcblk0p7 / mmcblk0p8)
|
||||||
|
RTOS_PART_MAX_BYTES = 4 * 1024 * 1024 # 4 MiB (mmcblk0p3 / mmcblk0p4)
|
||||||
|
|
||||||
|
CHUNK_SIZE = 1024 * 1024 # 1 MiB
|
||||||
|
|
||||||
|
DEFAULT_VERSION = "1.1.0.34"
|
||||||
|
DEFAULT_OPENKE_ARTIFACTS = Path("..") / "OpenKE" / "artifacts" / "buildroot-halley5-v30-image"
|
||||||
|
|
||||||
|
# Target profiles
|
||||||
|
# NOTE: Filenames must follow Creality's native pattern so master-server and local_ota_update.sh
|
||||||
|
# recognize them and find the extracted ota_config.in.
|
||||||
|
TARGET_PROFILES: Dict[str, Dict[str, str]] = {
|
||||||
|
"ke": {
|
||||||
|
"board_name": "F005",
|
||||||
|
"output_pattern": "Ender-3_V3_KE_F005_ota_img_V{version}.img",
|
||||||
|
"zero_bin": "assets/zero.bin",
|
||||||
|
"description": "Creality Ender-3 V3 KE (Target F005)",
|
||||||
|
},
|
||||||
|
"nebula": {
|
||||||
|
"board_name": "NEBULA",
|
||||||
|
"output_pattern": "NEBULA_ota_img_V{version}.img",
|
||||||
|
"zero_bin": "assets/zero_nebula.bin",
|
||||||
|
"description": "Creality Nebula Smart Kit / Nebula Pad (Target NEBULA)",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def derive_creality_password(board_name: str, salt: str = "cxswfile") -> str:
|
||||||
|
"""
|
||||||
|
Derives Creality's standard MD5-crypt password for a given board:
|
||||||
|
mkpasswd -m md5 "${BOARD_NAME}C3_7e_bz" -S cxswfile
|
||||||
|
Implemented in pure Python to eliminate platform dependencies.
|
||||||
|
"""
|
||||||
|
key = f"{board_name}C3_7e_bz"
|
||||||
|
magic = "$1$"
|
||||||
|
pw = key.encode("utf-8")
|
||||||
|
s = salt.encode("utf-8")
|
||||||
|
|
||||||
|
ctx = hashlib.md5(pw + magic.encode("utf-8") + s)
|
||||||
|
alt = hashlib.md5(pw + s + pw).digest()
|
||||||
|
|
||||||
|
for i in range(len(pw), 0, -16):
|
||||||
|
ctx.update(alt[: min(i, 16)])
|
||||||
|
|
||||||
|
i = len(pw)
|
||||||
|
while i > 0:
|
||||||
|
if i & 1:
|
||||||
|
ctx.update(b"\x00")
|
||||||
|
else:
|
||||||
|
ctx.update(pw[:1])
|
||||||
|
i >>= 1
|
||||||
|
|
||||||
|
digest = ctx.digest()
|
||||||
|
|
||||||
|
for idx in range(1000):
|
||||||
|
c = hashlib.md5()
|
||||||
|
if idx & 1:
|
||||||
|
c.update(pw)
|
||||||
|
else:
|
||||||
|
c.update(digest)
|
||||||
|
if idx % 3:
|
||||||
|
c.update(s)
|
||||||
|
if idx % 7:
|
||||||
|
c.update(pw)
|
||||||
|
if idx & 1:
|
||||||
|
c.update(digest)
|
||||||
|
else:
|
||||||
|
c.update(pw)
|
||||||
|
digest = c.digest()
|
||||||
|
|
||||||
|
b64 = "./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
|
||||||
|
|
||||||
|
def to64(v: int, n: int) -> str:
|
||||||
|
ret = []
|
||||||
|
for _ in range(n):
|
||||||
|
ret.append(b64[v & 0x3F])
|
||||||
|
v >>= 6
|
||||||
|
return "".join(ret)
|
||||||
|
|
||||||
|
res = magic + salt + "$"
|
||||||
|
d = digest
|
||||||
|
res += to64((d[0] << 16) | (d[6] << 8) | d[12], 4)
|
||||||
|
res += to64((d[1] << 16) | (d[7] << 8) | d[13], 4)
|
||||||
|
res += to64((d[2] << 16) | (d[8] << 8) | d[14], 4)
|
||||||
|
res += to64((d[3] << 16) | (d[9] << 8) | d[15], 4)
|
||||||
|
res += to64((d[4] << 16) | (d[10] << 8) | d[5], 4)
|
||||||
|
res += to64(d[11], 2)
|
||||||
|
return res
|
||||||
|
|
||||||
|
|
||||||
|
def compute_md5(path: Path) -> str:
|
||||||
|
"""Compute MD5 digest of a file in 4MB streaming chunks."""
|
||||||
|
digest = hashlib.md5()
|
||||||
|
with path.open("rb") as handle:
|
||||||
|
for chunk in iter(lambda: handle.read(4 * 1024 * 1024), b""):
|
||||||
|
digest.update(chunk)
|
||||||
|
return digest.hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def compute_sha256(path: Path) -> str:
|
||||||
|
"""Compute SHA256 digest of a file in 4MB streaming chunks."""
|
||||||
|
digest = hashlib.sha256()
|
||||||
|
with path.open("rb") as handle:
|
||||||
|
for chunk in iter(lambda: handle.read(4 * 1024 * 1024), b""):
|
||||||
|
digest.update(chunk)
|
||||||
|
return digest.hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def split_into_chunks(src_path: Path, dest_dir: Path, base_name: str) -> Tuple[str, List[str]]:
|
||||||
|
"""
|
||||||
|
Split payload into 1 MiB chunks per Creality OTA specification:
|
||||||
|
<base_name>.<index:04d>.<prev_md5>
|
||||||
|
where index 0000 uses the full file MD5 as prev_md5.
|
||||||
|
Returns (full_file_md5, list_of_chunk_md5s).
|
||||||
|
"""
|
||||||
|
full_md5 = compute_md5(src_path)
|
||||||
|
chunk_md5s: List[str] = []
|
||||||
|
|
||||||
|
with src_path.open("rb") as handle:
|
||||||
|
index = 0
|
||||||
|
prev_md5 = full_md5
|
||||||
|
while True:
|
||||||
|
data = handle.read(CHUNK_SIZE)
|
||||||
|
if not data:
|
||||||
|
break
|
||||||
|
chunk_md5 = hashlib.md5(data).hexdigest()
|
||||||
|
chunk_name = f"{base_name}.{index:04d}.{prev_md5}"
|
||||||
|
chunk_path = dest_dir / chunk_name
|
||||||
|
chunk_path.write_bytes(data)
|
||||||
|
chunk_md5s.append(chunk_md5)
|
||||||
|
prev_md5 = chunk_md5
|
||||||
|
index += 1
|
||||||
|
|
||||||
|
return full_md5, chunk_md5s
|
||||||
|
|
||||||
|
|
||||||
|
def verify_manifest(manifest_path: Path, ximage_path: Path, rootfs_path: Path) -> None:
|
||||||
|
"""Verify xImage and rootfs.squashfs match build-manifest.txt if present."""
|
||||||
|
if not manifest_path.exists():
|
||||||
|
print(f"[*] Manifest {manifest_path} not found, skipping manifest verification.")
|
||||||
|
return
|
||||||
|
|
||||||
|
print(f"[*] Verifying inputs against build manifest: {manifest_path}")
|
||||||
|
manifest_data = {}
|
||||||
|
for line in manifest_path.read_text(encoding="utf-8").splitlines():
|
||||||
|
line = line.strip()
|
||||||
|
if "=" in line and not line.startswith("#"):
|
||||||
|
key, val = line.split("=", 1)
|
||||||
|
manifest_data[key.strip()] = val.strip()
|
||||||
|
|
||||||
|
expected_ximage_size = manifest_data.get("xImage_size")
|
||||||
|
expected_ximage_sha = manifest_data.get("xImage_sha256")
|
||||||
|
expected_rootfs_size = manifest_data.get("rootfs_squashfs_size")
|
||||||
|
expected_rootfs_sha = manifest_data.get("rootfs_squashfs_sha256")
|
||||||
|
|
||||||
|
if expected_ximage_size and ximage_path.stat().st_size != int(expected_ximage_size):
|
||||||
|
raise ValueError(
|
||||||
|
f"xImage size mismatch: {ximage_path.stat().st_size} bytes vs expected {expected_ximage_size}"
|
||||||
|
)
|
||||||
|
if expected_ximage_sha:
|
||||||
|
actual_ximage_sha = compute_sha256(ximage_path)
|
||||||
|
if actual_ximage_sha != expected_ximage_sha:
|
||||||
|
raise ValueError(f"xImage SHA256 mismatch: {actual_ximage_sha} != {expected_ximage_sha}")
|
||||||
|
|
||||||
|
if expected_rootfs_size and rootfs_path.stat().st_size != int(expected_rootfs_size):
|
||||||
|
raise ValueError(
|
||||||
|
f"rootfs.squashfs size mismatch: {rootfs_path.stat().st_size} bytes vs expected {expected_rootfs_size}"
|
||||||
|
)
|
||||||
|
if expected_rootfs_sha:
|
||||||
|
actual_rootfs_sha = compute_sha256(rootfs_path)
|
||||||
|
if actual_rootfs_sha != expected_rootfs_sha:
|
||||||
|
raise ValueError(
|
||||||
|
f"rootfs.squashfs SHA256 mismatch: {actual_rootfs_sha} != {expected_rootfs_sha}"
|
||||||
|
)
|
||||||
|
|
||||||
|
print(" [+] Manifest verification passed: exact size & SHA256 match.")
|
||||||
|
|
||||||
|
|
||||||
|
def pack_archive(staging_root: Path, archive_name: str, output_img: Path, password: str) -> None:
|
||||||
|
"""Create encrypted 7z archive using 7z CLI (if available) or py7zr."""
|
||||||
|
output_img.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
if output_img.exists():
|
||||||
|
output_img.unlink()
|
||||||
|
|
||||||
|
has_7z = shutil.which("7z") is not None
|
||||||
|
|
||||||
|
if has_7z:
|
||||||
|
print("[*] Compressing encrypted 7z archive using system 7z CLI (multi-threaded)...")
|
||||||
|
cmd = [
|
||||||
|
"7z", "a",
|
||||||
|
"-t7z",
|
||||||
|
f"-p{password}",
|
||||||
|
"-mhe=on",
|
||||||
|
"-mx=4",
|
||||||
|
str(output_img),
|
||||||
|
archive_name,
|
||||||
|
]
|
||||||
|
res = subprocess.run(cmd, cwd=str(staging_root.parent), stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
|
||||||
|
if res.returncode != 0:
|
||||||
|
raise RuntimeError(f"7z command failed with code {res.returncode}:\n{res.stdout}")
|
||||||
|
else:
|
||||||
|
print("[*] System 7z not found. Compressing using Python py7zr (single-threaded)...")
|
||||||
|
try:
|
||||||
|
import py7zr
|
||||||
|
except ImportError:
|
||||||
|
raise RuntimeError("Neither 7z CLI nor python 'py7zr' is available. Install 7zip or pip install py7zr.")
|
||||||
|
|
||||||
|
with py7zr.SevenZipFile(output_img, mode="w", password=password, header_encryption=True) as zf:
|
||||||
|
zf.writeall(staging_root, arcname=archive_name)
|
||||||
|
|
||||||
|
|
||||||
|
def verify_archive(archive_path: Path, password: str) -> None:
|
||||||
|
"""Verify archive integrity and password decryption."""
|
||||||
|
print(f"[*] Verifying archive integrity: {archive_path}")
|
||||||
|
has_7z = shutil.which("7z") is not None
|
||||||
|
if has_7z:
|
||||||
|
cmd = ["7z", "t", f"-p{password}", str(archive_path)]
|
||||||
|
res = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
|
||||||
|
if res.returncode != 0:
|
||||||
|
raise RuntimeError(f"Archive verification failed:\n{res.stderr or res.stdout}")
|
||||||
|
else:
|
||||||
|
import py7zr
|
||||||
|
with py7zr.SevenZipFile(archive_path, mode="r", password=password) as zf:
|
||||||
|
if not zf.test():
|
||||||
|
raise RuntimeError("Archive validation failed via py7zr.")
|
||||||
|
print(" [+] Archive verification passed: 100% valid encrypted 7z envelope.")
|
||||||
|
|
||||||
|
|
||||||
|
def build_openke_ota_image(
|
||||||
|
ximage_path: Path,
|
||||||
|
rootfs_path: Path,
|
||||||
|
zero_path: Path,
|
||||||
|
manifest_path: Path | None,
|
||||||
|
output_img: Path,
|
||||||
|
version: str,
|
||||||
|
password: str,
|
||||||
|
target_desc: str = "Creality Display",
|
||||||
|
) -> Path:
|
||||||
|
"""End-to-end packaging function for OpenKE OTA .img."""
|
||||||
|
print("=" * 65)
|
||||||
|
print(f" OpenKE Firmware Packager: {target_desc}")
|
||||||
|
print("=" * 65)
|
||||||
|
|
||||||
|
if not ximage_path.exists():
|
||||||
|
raise FileNotFoundError(f"xImage not found: {ximage_path}")
|
||||||
|
if not rootfs_path.exists():
|
||||||
|
raise FileNotFoundError(f"rootfs.squashfs not found: {rootfs_path}")
|
||||||
|
if not zero_path.exists():
|
||||||
|
raise FileNotFoundError(f"zero.bin (RTOS firmware) not found at {zero_path}.")
|
||||||
|
|
||||||
|
ximage_size = ximage_path.stat().st_size
|
||||||
|
rootfs_size = rootfs_path.stat().st_size
|
||||||
|
zero_size = zero_path.stat().st_size
|
||||||
|
|
||||||
|
print("[*] Partition Budget Preflight:")
|
||||||
|
ximage_pct = (ximage_size / KERNEL_PART_MAX_BYTES) * 100
|
||||||
|
ximage_free = (KERNEL_PART_MAX_BYTES - ximage_size) / (1024 * 1024)
|
||||||
|
print(f" - Kernel: {ximage_size:,} B / {KERNEL_PART_MAX_BYTES:,} B ({ximage_pct:.1f}% used, {ximage_free:.2f} MiB free)")
|
||||||
|
if ximage_size > KERNEL_PART_MAX_BYTES:
|
||||||
|
raise ValueError(f"xImage ({ximage_size} B) exceeds max partition size ({KERNEL_PART_MAX_BYTES} B)!")
|
||||||
|
|
||||||
|
rootfs_pct = (rootfs_size / ROOTFS_PART_MAX_BYTES) * 100
|
||||||
|
rootfs_free = (ROOTFS_PART_MAX_BYTES - rootfs_size) / (1024 * 1024)
|
||||||
|
print(f" - Rootfs: {rootfs_size:,} B / {ROOTFS_PART_MAX_BYTES:,} B ({rootfs_pct:.1f}% used, {rootfs_free:.2f} MiB free)")
|
||||||
|
if rootfs_size > ROOTFS_PART_MAX_BYTES:
|
||||||
|
raise ValueError(f"rootfs.squashfs ({rootfs_size} B) exceeds max partition size ({ROOTFS_PART_MAX_BYTES} B)!")
|
||||||
|
|
||||||
|
print(f" - RTOS: {zero_size:,} B / {RTOS_PART_MAX_BYTES:,} B")
|
||||||
|
if zero_size > RTOS_PART_MAX_BYTES:
|
||||||
|
raise ValueError(f"zero.bin ({zero_size} B) exceeds max partition size ({RTOS_PART_MAX_BYTES} B)!")
|
||||||
|
|
||||||
|
if manifest_path:
|
||||||
|
verify_manifest(manifest_path, ximage_path, rootfs_path)
|
||||||
|
|
||||||
|
# CRITICAL: Creality's local_ota_update.sh expects:
|
||||||
|
# OTA_UNZIP_FILE_NAME=${OTA_FILE_NAME%.img*}
|
||||||
|
# ota_site=${OTA_FILE_PATH}/${OTA_UNZIP_FILE_NAME}
|
||||||
|
# ota_site_config=$ota_site/ota_config.in
|
||||||
|
# Therefore the root directory inside the archive MUST exactly equal output_img.stem!
|
||||||
|
archive_name = output_img.name
|
||||||
|
if archive_name.endswith(".img"):
|
||||||
|
archive_name = archive_name[:-4]
|
||||||
|
|
||||||
|
with tempfile.TemporaryDirectory(prefix="openke_ota_stage_") as tmpdir:
|
||||||
|
stage_dir = Path(tmpdir)
|
||||||
|
staging_root = stage_dir / archive_name
|
||||||
|
version_dir = staging_root / f"ota_v{version}"
|
||||||
|
version_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
print(f"[*] Packaging archive layout: {archive_name}/ota_v{version}/")
|
||||||
|
print("[*] Slicing and chunking payloads (1 MiB slices):")
|
||||||
|
print(" - Slicing xImage...")
|
||||||
|
ximage_md5, ximage_chunk_md5s = split_into_chunks(ximage_path, version_dir, "xImage")
|
||||||
|
print(f" Total chunks: {len(ximage_chunk_md5s)}, Full MD5: {ximage_md5}")
|
||||||
|
|
||||||
|
print(" - Slicing rootfs.squashfs...")
|
||||||
|
rootfs_md5, rootfs_chunk_md5s = split_into_chunks(rootfs_path, version_dir, "rootfs.squashfs")
|
||||||
|
print(f" Total chunks: {len(rootfs_chunk_md5s)}, Full MD5: {rootfs_md5}")
|
||||||
|
|
||||||
|
print(" - Slicing zero.bin...")
|
||||||
|
zero_md5, zero_chunk_md5s = split_into_chunks(zero_path, version_dir, "zero.bin")
|
||||||
|
print(f" Total chunks: {len(zero_chunk_md5s)}, Full MD5: {zero_md5}")
|
||||||
|
|
||||||
|
print("[*] Generating Creality OTA manifests and MD5 sidecars...")
|
||||||
|
(staging_root / "ota_config.in").write_text(f"current_version={version}\n", encoding="utf-8", newline="\n")
|
||||||
|
(version_dir / f"ota_v{version}.ok").write_bytes(b"\n")
|
||||||
|
|
||||||
|
ota_update_lines = [
|
||||||
|
f"ota_version={version}",
|
||||||
|
"",
|
||||||
|
"img_type=kernel",
|
||||||
|
"img_name=xImage",
|
||||||
|
f"img_size={ximage_size}",
|
||||||
|
f"img_md5={ximage_md5}",
|
||||||
|
"",
|
||||||
|
"img_type=rootfs",
|
||||||
|
"img_name=rootfs.squashfs",
|
||||||
|
f"img_size={rootfs_size}",
|
||||||
|
f"img_md5={rootfs_md5}",
|
||||||
|
"",
|
||||||
|
"img_type=rtos",
|
||||||
|
"img_name=zero.bin",
|
||||||
|
f"img_size={zero_size}",
|
||||||
|
f"img_md5={zero_md5}",
|
||||||
|
"",
|
||||||
|
]
|
||||||
|
(version_dir / "ota_update.in").write_text("\n".join(ota_update_lines) + "\n", encoding="utf-8", newline="\n")
|
||||||
|
|
||||||
|
(version_dir / f"ota_md5_xImage.{ximage_md5}").write_text("\n".join(ximage_chunk_md5s) + "\n", encoding="utf-8", newline="\n")
|
||||||
|
(version_dir / f"ota_md5_rootfs.squashfs.{rootfs_md5}").write_text("\n".join(rootfs_chunk_md5s) + "\n", encoding="utf-8", newline="\n")
|
||||||
|
(version_dir / f"ota_md5_zero.bin.{zero_md5}").write_text("\n".join(zero_chunk_md5s) + "\n", encoding="utf-8", newline="\n")
|
||||||
|
|
||||||
|
pack_archive(staging_root, archive_name, output_img, password)
|
||||||
|
|
||||||
|
verify_archive(output_img, password)
|
||||||
|
|
||||||
|
print("=" * 65)
|
||||||
|
print(" SUCCESS: OpenKE OTA Image Created Successfully!")
|
||||||
|
print(f" Target Image: {output_img}")
|
||||||
|
print(f" Size: {output_img.stat().st_size:,} bytes ({output_img.stat().st_size / (1024*1024):.2f} MiB)")
|
||||||
|
print(f" Package Ver: {version}")
|
||||||
|
print("=" * 65)
|
||||||
|
print(" FLASHING INSTRUCTIONS:")
|
||||||
|
print(" 1. Format a USB flash drive as FAT32.")
|
||||||
|
print(f" 2. Copy '{output_img.name}' to the root of the USB drive.")
|
||||||
|
print(" 3. Insert the USB drive into the printer / Nebula Pad front USB port.")
|
||||||
|
print(" 4. On the touchscreen: Settings -> Firmware Update -> Local Update.")
|
||||||
|
print(" 5. Select the update and confirm. The device will flash OpenKE to")
|
||||||
|
print(" Slot 2, point the bootloader to ota:kernel2, and automatically reboot.")
|
||||||
|
print(" 6. First boot will seed /usr/data/openke and launch Klipper & Moonraker.")
|
||||||
|
print("=" * 65)
|
||||||
|
return output_img
|
||||||
|
|
||||||
|
|
||||||
|
def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
description="Package OpenKE build artifacts into stock CrealityOS-compatible USB update (.img) packages."
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--target",
|
||||||
|
choices=["ke", "nebula", "all"],
|
||||||
|
default="ke",
|
||||||
|
help="Target hardware profile: 'ke' (Ender-3 V3 KE), 'nebula' (Nebula Pad), or 'all' (build both) (default: ke)",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--artifacts-dir",
|
||||||
|
type=Path,
|
||||||
|
default=None,
|
||||||
|
help="Path to OpenKE build artifacts directory (contains xImage, rootfs.squashfs, and build-manifest.txt)",
|
||||||
|
)
|
||||||
|
parser.add_argument("--ximage", type=Path, default=None, help="Explicit path to OpenKE xImage")
|
||||||
|
parser.add_argument("--rootfs", type=Path, default=None, help="Explicit path to OpenKE rootfs.squashfs")
|
||||||
|
parser.add_argument("--manifest", type=Path, default=None, help="Explicit path to build-manifest.txt")
|
||||||
|
parser.add_argument(
|
||||||
|
"--zero-bin",
|
||||||
|
type=Path,
|
||||||
|
default=None,
|
||||||
|
help="Path to RTOS firmware (zero.bin). Defaults to profile asset.",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--version",
|
||||||
|
default=DEFAULT_VERSION,
|
||||||
|
help=f"Firmware package version recognized by stock updater (default: {DEFAULT_VERSION})",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--output",
|
||||||
|
type=Path,
|
||||||
|
default=None,
|
||||||
|
help="Output image path (applicable when packaging a single target)",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--password",
|
||||||
|
default=None,
|
||||||
|
help="Custom Creality 7z encryption password (defaults to derived per-board key)",
|
||||||
|
)
|
||||||
|
return parser.parse_args(argv)
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: Sequence[str] | None = None) -> int:
|
||||||
|
args = parse_args(argv)
|
||||||
|
|
||||||
|
artifacts_dir = args.artifacts_dir
|
||||||
|
if artifacts_dir is None and (args.ximage is None or args.rootfs is None):
|
||||||
|
if DEFAULT_OPENKE_ARTIFACTS.exists():
|
||||||
|
artifacts_dir = DEFAULT_OPENKE_ARTIFACTS
|
||||||
|
|
||||||
|
ximage = args.ximage
|
||||||
|
rootfs = args.rootfs
|
||||||
|
manifest = args.manifest
|
||||||
|
|
||||||
|
if artifacts_dir:
|
||||||
|
artifacts_dir = artifacts_dir.resolve()
|
||||||
|
if ximage is None:
|
||||||
|
ximage = artifacts_dir / "xImage"
|
||||||
|
if rootfs is None:
|
||||||
|
rootfs = artifacts_dir / "rootfs.squashfs"
|
||||||
|
if manifest is None and (artifacts_dir / "build-manifest.txt").exists():
|
||||||
|
manifest = artifacts_dir / "build-manifest.txt"
|
||||||
|
|
||||||
|
if ximage is None or rootfs is None:
|
||||||
|
print("Error: Must provide --artifacts-dir OR both --ximage and --rootfs", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
ximage = ximage.resolve()
|
||||||
|
rootfs = rootfs.resolve()
|
||||||
|
if manifest:
|
||||||
|
manifest = manifest.resolve()
|
||||||
|
|
||||||
|
version = args.version
|
||||||
|
targets_to_build = ["ke", "nebula"] if args.target == "all" else [args.target]
|
||||||
|
|
||||||
|
if args.output and len(targets_to_build) > 1:
|
||||||
|
print("Error: --output cannot be used with --target all (use default paths).", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
for tgt in targets_to_build:
|
||||||
|
profile = TARGET_PROFILES[tgt]
|
||||||
|
board_name = profile["board_name"]
|
||||||
|
desc = profile["description"]
|
||||||
|
|
||||||
|
password = args.password
|
||||||
|
if password is None:
|
||||||
|
password = derive_creality_password(board_name)
|
||||||
|
|
||||||
|
zero_bin = args.zero_bin
|
||||||
|
if zero_bin is None:
|
||||||
|
script_dir = Path(__file__).parent.parent
|
||||||
|
zero_bin = script_dir / profile["zero_bin"]
|
||||||
|
zero_bin = zero_bin.resolve()
|
||||||
|
|
||||||
|
output = args.output
|
||||||
|
if output is None:
|
||||||
|
out_filename = profile["output_pattern"].format(version=version)
|
||||||
|
output = Path("build") / out_filename
|
||||||
|
output = output.resolve()
|
||||||
|
|
||||||
|
try:
|
||||||
|
build_openke_ota_image(
|
||||||
|
ximage_path=ximage,
|
||||||
|
rootfs_path=rootfs,
|
||||||
|
zero_path=zero_bin,
|
||||||
|
manifest_path=manifest,
|
||||||
|
output_img=output,
|
||||||
|
version=version,
|
||||||
|
password=password,
|
||||||
|
target_desc=desc,
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
print(f"\n[!] ERROR ({tgt}): {exc}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
Reference in New Issue
Block a user