Initial Commit

Fimeware Pre-Rooted, Builds & Flashes
This commit is contained in:
Dark98
2026-07-12 18:19:18 +00:00
commit c0dd058ae7
11 changed files with 1151 additions and 0 deletions
+4
View File
@@ -0,0 +1,4 @@
# Generated firmware artifacts
/build/
tools/__pycache__/
**/__pycache__/
Binary file not shown.
+50
View File
@@ -0,0 +1,50 @@
# Firmware Changelog
Differences between the stock Creality Ender-3 V3 KE image and our custom build.
## Baseline
- Stock image: `Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img`
- Custom build line: `DarKE_KE_F005_ota_img_V11034.img`
## Firmware Changes
- Added `overlay_rootfs/etc/init.d/S05agree_root` to create `/usr/data/creality/userdata/user_agree_root` at boot.
- Updated the version metadata used by the UI:
- `overlay_rootfs/etc/ota_info`
- `overlay_rootfs/usr/share/klipper/config/F005/printer.cfg`
- Current display version: `v1.1.0.33`
## Package Identity
- OTA package version: `11034`
- UI/display version: `v1.1.0.33`
- The package still flashes through the stock USB updater flow.
## Build System Changes
- `tools/ke_firmware.py` now:
- extracts the stock OTA archive
- unpacks the stock SquashFS rootfs
- applies the overlay
- rebuilds the rootfs
- repacks the archive
- Repacking preserves stock metadata:
- symlinks stay symlinks
- executable bits stay intact
- ownership and timestamps are preserved where applicable
- Added archive validation and stock-vs-rebuilt compare support.
- New `etc/init.d/*` overlay scripts are forced to pack as executable.
## Verified
- The rebuilt image matches stock rootfs metadata outside the overlay.
- The custom image rebuilds cleanly and flashes successfully.
- The root-agree file is created at boot by init, not baked in as a static overlay file.
# Changelog:
## 0.0.1
Pre-Rooted
+69
View File
@@ -0,0 +1,69 @@
# Ender-3 V3 KE Firmware Notes
## Stock firmware source
- Downloaded official image: `Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img`
- Official version: `V1.1.0.17`
- Manifest layers inside the archive:
- `xImage` (`kernel`)
- `rootfs.squashfs` (`rootfs`)
- `zero.bin` (`rtos`)
## 7z archive password
The official Creality firmware archive is password-protected.
Password:
`$1$cxswfile$ZFd0RWFYkJQugbtKVGL9y0`
## Workspace layout
- Stock archive cache: `./Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img`
- Overlay tree for our changes: `./overlay_rootfs/`
- Build cache: `./build/_toolcache/squashfs-tools-ng-1.3.2-mingw64/`
- Temporary build workspace: whichever directory you pass with `--build-dir`
## Useful edit points in the stock rootfs
- Klipper startup/service hook: `rootfs_extract/etc/init.d/S55klipper_service`
- WebRTC startup/service hook: `rootfs_extract/etc/init.d/S97webrtc`
- Stock printer profile:
- `rootfs_extract/usr/share/klipper/config/F005/printer.cfg`
- `rootfs_extract/usr/share/klipper/config/F005/gcode_macro.cfg`
- `rootfs_extract/usr/share/klipper/config/F005/printer_params.cfg`
- Runtime printer config is copied into `/usr/data/printer_data/config/` on the device
## Notes
- The shipped `.img` file is a 7z archive, not a raw flash image.
- The build flow starts from the stock archive, overlays `overlay_rootfs/`, and repacks the result.
- The helper uses native `rdsquashfs` and `gensquashfs` on Linux. Install
`squashfs-tools-ng` so those binaries are on `PATH`. On Windows it uses the
bundled `rdsquashfs.exe` / `gensquashfs.exe` tools from `squashfs-tools-ng`.
- The repack path is metadata-preserving: it reuses the stock SquashFS layout so symlinks and execute bits survive the round trip.
- `xImage.full` is a stock U-Boot `uImage` kernel blob. Its raw payload can be derived if kernel-side edits are needed.
- Stock-updater-facing version scheme in this workspace: `11034`
- UI/display version stays on the dotted printer-side label `v1.1.0.33`
- The display/version string is sourced from `overlay_rootfs/etc/ota_info`.
- The firmware image is intended to flash through the stock USB updater without any manual printer-side file edits.
- The helper build process keeps the stock updater happy by using the numeric package version and stock archive layout.
## Rebuild
Use the helper script:
```powershell
python .\tools\ke_firmware.py build --build-dir D:\DarKE_build --stock-archive .\Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img --overlay-dir .\overlay_rootfs --output .\build\DarKE_KE_F005_ota_img_V11034.img
```
Compare the rebuilt rootfs against stock, allowing only the overlay tree to differ:
```powershell
python .\tools\ke_firmware.py compare --stock-archive .\Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img --rebuilt-archive .\build\DarKE_KE_F005_ota_img_V11034.img --overlay-dir .\overlay_rootfs
```
## Stock-safe flash summary
- Put the generated `DarKE_KE_F005_ota_img_V11034.img` on a USB drive.
- Flash it with the printer's built-in updater.
+49
View File
@@ -0,0 +1,49 @@
# Ender-3 V3 KE firmware workspace
This repo contains a stock KE firmware image, a small overlay tree of our changes, and a helper script for rebuilding an `.img` package.
## Files
- `FIRMWARE_NOTES.md`: archive password and firmware layout notes
- `tools/ke_firmware.py`: extract/rebuild helper
- `requirements.txt`: Python dependency list for the helper script
## Python setup
Install the helper dependency with:
```bash
pip install -r requirements.txt
```
On Linux, install `squashfs-tools-ng` so `gensquashfs` and `rdsquashfs` are
available on `PATH`. On Windows, the helper uses the bundled `.exe` tools.
## Rebuild flow
```powershell
python .\tools\ke_firmware.py build --stock-archive .\Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img --overlay-dir .\overlay_rootfs --output .\build\DarKE_KE_F005_ota_img_V11034.img
```
If the stock archive is missing, pass `--stock-url` and the helper will download it before extracting.
If your `C:` drive is tight, point `--build-dir` at a larger drive so the stock rootfs can be unpacked there.
The build step downloads or reuses the stock archive, extracts it, overlays `overlay_rootfs/`, rebuilds the root filesystem with the Windows `squashfs-tools-ng` binaries, and repacks the Creality image envelope with `py7zr`.
The rebuild is metadata-preserving, so stock symlinks and executable bits survive the round trip.
## Compare flow
```powershell
python .\tools\ke_firmware.py compare --stock-archive .\Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img --rebuilt-archive .\build\DarKE_KE_F005_ota_img_V11034.img --overlay-dir .\overlay_rootfs
```
Use this after a build to confirm the rebuilt rootfs still matches stock everywhere except the overlayed files.
## Stock flash flow
1. Copy the generated `.img` to a USB drive.
2. Insert the USB drive into the Ender-3 V3 KE.
3. Use the built-in updater to flash the image.
4. After reboot, the firmware reports the UI/display version from the package metadata.
This flow does not require manual edits on the printer itself.
+1
View File
@@ -0,0 +1 @@
etc/init.d/S98sync_version
+12
View File
@@ -0,0 +1,12 @@
#!/bin/sh
AGREE_ROOT_FILE=/usr/data/creality/userdata/user_agree_root
case "$1" in
start)
mkdir -p /usr/data/creality/userdata
: > "$AGREE_ROOT_FILE"
;;
esac
true
+4
View File
@@ -0,0 +1,4 @@
ota_version=1.1.0.33
ota_board_name=F005
ota_compile_time=2024 08.06 09:12:56
ota_site=http://192.168.43.52/ota/board_test
@@ -0,0 +1,256 @@
# Ender-3V3KE
# Printer_size: 220x220x240
# Version: v1.1.0.33
# CreateDate: 2024/3/27
# mcu: chip: GD32F303RET6
# version: CR4NS200323C10
[include sensorless.cfg]
[include gcode_macro.cfg]
[include printer_params.cfg]
[mcu]
serial:/dev/ttyS1
baud:230400
restart_method: command
[force_move]
enable_force_move: True
[mcu rpi]
serial: /tmp/klipper_host_mcu
[bl24c16f]
i2c_mcu: rpi
i2c_bus: i2c.2
i2c_speed: 400000
######################################################
[prtouch_v2]
pres_cnt: 1 #探点次数
pres0_clk_pins: PA4 #压力检测时钟引脚配置
pres0_sdo_pins: PC6 #压力检测数据引脚配置
step_swap_pin: PA15
pres_swap_pin: PA15
step_base:2
# show_msg: True
tri_min_hold: 1000
tri_max_hold: 1500 #压力检测信息展示
speed: 1
# tri_wave_ip: 172.22.30.204
#####################################################
[z_compensate]
tri_min_hold: 1400
tri_max_hold: 2000 #压力检测信息展示
tri_expand_mm = 0.10
# tri_min_hold: 3
speed: 5
hot_start_temp: 180#擦喷嘴是最小温度
hot_rub_temp: 200#擦喷嘴是最小温度
hot_end_temp: 140#擦喷嘴是最小温度
bed_add_temp: 60#调平时的热床温度
clr_noz_start_x: -3 #擦喷头区域的起始x坐标(默认在热床后方正中心位置)
clr_noz_start_y: 20 #擦喷头区域的起始y坐标
clr_noz_len_x: 3 #擦喷头区域的x方向的长度
clr_noz_len_y: 50 #擦喷头区域的y方向的长度
pa_clr_dis_mm_x = 0
pa_clr_dis_mm_y =30
# show_msg = True
bl_offset: 0,27
noz_pos_center: 20,25
noz_pos_offset: 3,7
pumpback_mm: 10
vs_start_z_pos: 3
pr_probe_cnt: 3
pr_clear_probe_cnt: 3
type_nozz = 0
[printer]
kinematics: cartesian
max_velocity: 500
max_accel: 8000
max_accel_to_decel: 2500
max_z_velocity: 30
square_corner_velocity: 5.0
max_z_accel: 300
[idle_timeout]
timeout: 99999999
[stepper_x]
step_pin: PC2
dir_pin: !PB9
enable_pin: !PC3
microsteps: 16
rotation_distance: 40
endstop_pin: !PA5
position_endstop: -12
position_min: -12
position_max: 221
homing_speed: 30
homing_retract_dist:0 #10
[tmc2208 stepper_x]
uart_pin:PB12
interpolate: True
run_current:0.75
sense_resistor: 0.150
stealthchop_threshold: 0
[stepper_y]
step_pin: PB8
dir_pin: PB7
enable_pin: !PC3
microsteps: 16
rotation_distance: 60
endstop_pin: !PA6
position_endstop: -20
position_min: -20
position_max: 223
homing_speed: 30
homing_retract_dist:0
[tmc2208 stepper_y]
uart_pin:PB13
interpolate: True
run_current:0.75
sense_resistor: 0.150
stealthchop_threshold: 0
[stepper_z]
step_pin: PB6
dir_pin: !PB5
enable_pin: !PC3
microsteps: 16
rotation_distance:8
endstop_pin:probe:z_virtual_endstop#PA15 #probe:z_virtual_endstop
position_max: 246
position_min: -5
[tmc2208 stepper_z]
uart_pin: PB14
interpolate: True
run_current: 0.8
stealthchop_threshold: 0
sense_resistor: 0.150
[bltouch]
sensor_pin:PC14
control_pin: PC13
x_offset: 0
y_offset: 27
z_offset: 0
probe_with_touch_mode: true
stow_on_each_sample: false
speed:5
lift_speed:20
[filament_switch_sensor filament_sensor]
switch_pin: !PC15
pause_on_runout: true
[output_pin MainBoardFan]
pin: !PB1
[extruder]
max_extrude_only_distance:1000
max_extrude_cross_section:80
pressure_advance = 0.036
step_pin: PB4
dir_pin: PB3
enable_pin: !PC3
microsteps: 16
rotation_distance: 7.53
nozzle_diameter: 0.400
filament_diameter: 1.750
heater_pin: PA1
sensor_type: EPCOS 100K B57560G104F
sensor_pin: PC5
control = pid
pid_Kp=20.584
pid_Ki=1.737
pid_Kd=60.981
min_temp: 0
max_temp: 320 # Set to 300 for S1 Pro
[heater_bed]
heater_pin: PB2
sensor_type: EPCOS 100K B57560G104F
sensor_pin: PC4
control = pid
pid_kp = 70.652
pid_ki = 1.798
pid_kd = 694.157
min_temp: 0
max_temp: 120 #
temp_offset_flag = True
[verify_heater extruder]
[verify_heater heater_bed]
check_gain_time: 120
heating_gain: 1.0
hysteresis: 10
[temperature_sensor mcu_temp]
sensor_type: temperature_mcu
min_temp: 0
max_temp: 100
[output_pin fan0]
pin:PA0
pwm: True
cycle_time: 0.0100
hardware_pwm: false
value: 0.00
scale: 255
shutdown_value: 0.0
[heater_fan nozzle_fan]
pin: PC1
max_power: 1.0
shutdown_speed: 0
cycle_time: 0.010
hardware_pwm: False
kick_start_time: 0.100
off_below: 0.0
heater: extruder
fan_speed: 1.0
heater_temp: 60.0
[bed_mesh]
speed: 350
mesh_min: 5,10 #need to handle head distance with bl_touch
mesh_max: 215,215 #max probe range
probe_count: 5,5
fade_start: 1
fade_end: 10
fade_target: 0
horizontal_move_z: 8
[input_shaper]
shaper_type_y = mzv
shaper_freq_y = 32.0
shaper_type_x = mzv
shaper_freq_x = 38.5
[mcu rpi]
serial: /tmp/klipper_host_mcu
[adxl345]
cs_pin: rpi:None
spi_speed: 2000000
spi_bus: spidev2.0
axes_map: z,y,x
[resonance_tester]
accel_chip: adxl345
accel_per_hz: 70
probe_points: 117.5,117.5,100
max_freq: 90
+1
View File
@@ -0,0 +1 @@
py7zr
+705
View File
@@ -0,0 +1,705 @@
#!/usr/bin/env python3
"""
Ender-3 V3 KE firmware extractor/repacker.
This script handles the stock Creality 7z-wrapped `.img` package, reconstructs
the split payload files, and can rebuild a fresh package from an extracted
rootfs tree.
"""
from __future__ import annotations
import argparse
import hashlib
import os
import shlex
import re
import shutil
import subprocess
import sys
import tempfile
import urllib.request
import zipfile
from pathlib import Path
from typing import List, Sequence, Tuple
import py7zr
PASSWORD = "$1$cxswfile$ZFd0RWFYkJQugbtKVGL9y0"
CHUNK_SIZE = 1024 * 1024
UIMAGE_HEADER_SIZE = 64
DEFAULT_STOCK_ARCHIVE = Path("Ender-3_V3_KE_F005_ota_img_V1.1.0.17.img")
DEFAULT_OVERLAY_DIR = Path("overlay_rootfs")
DEFAULT_VERSION = "11034"
SQUASHFS_TOOLS_URL = "https://infraroot.at/pub/squashfs/windows/squashfs-tools-ng-1.3.2-mingw64.zip"
SQUASHFS_TOOLS_DIR = Path("build") / "_toolcache" / "squashfs-tools-ng-1.3.2-mingw64"
def md5_file(path: Path) -> str:
digest = hashlib.md5()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(4 * 1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def ensure_clean_dir(path: Path) -> None:
if path.exists():
shutil.rmtree(path)
path.mkdir(parents=True, exist_ok=True)
def reconstruct_split_file(src_dir: Path, prefix: str, out_path: Path) -> List[Path]:
pattern = re.compile(rf"^{re.escape(prefix)}\.(\d{{4}})\.[0-9a-f]{{32}}$")
parts: List[Tuple[int, Path]] = []
for entry in src_dir.iterdir():
if not entry.is_file():
continue
match = pattern.match(entry.name)
if match:
parts.append((int(match.group(1)), entry))
if not parts:
raise FileNotFoundError(f"no split files found for {prefix} in {src_dir}")
parts.sort(key=lambda item: item[0])
with out_path.open("wb") as out_handle:
for _, part in parts:
out_handle.write(part.read_bytes())
return [path for _, path in parts]
def write_lines(path: Path, lines: Sequence[str]) -> None:
path.write_text("\n".join(lines) + "\n", encoding="utf-8", newline="\n")
def split_file(src_path: Path, dest_dir: Path, base_name: str) -> Tuple[str, List[Path], List[str]]:
full_md5 = md5_file(src_path)
chunk_md5s: List[str] = []
chunk_paths: List[Path] = []
with src_path.open("rb") as handle:
index = 0
previous_md5 = full_md5
while True:
data = handle.read(CHUNK_SIZE)
if not data:
break
chunk_md5 = hashlib.md5(data).hexdigest()
chunk_name = f"{base_name}.{index:04d}.{previous_md5}"
chunk_path = dest_dir / chunk_name
chunk_path.write_bytes(data)
chunk_paths.append(chunk_path)
chunk_md5s.append(chunk_md5)
previous_md5 = chunk_md5
index += 1
return full_md5, chunk_paths, chunk_md5s
def download_file(url: str, dest: Path) -> None:
dest.parent.mkdir(parents=True, exist_ok=True)
temp_dest = dest.with_name(dest.name + ".download")
if temp_dest.exists():
temp_dest.unlink()
with urllib.request.urlopen(url) as response, temp_dest.open("wb") as out_handle:
shutil.copyfileobj(response, out_handle)
temp_dest.replace(dest)
def resolve_squashfs_tool(tool_name: str) -> Path:
"""
Resolve a squashfs-tools-ng executable for the current host.
On Windows we use the cached MinGW bundle that ships with this repo.
On other platforms we require a native binary from PATH.
"""
if os.name == "nt":
tools_bin = ensure_squashfs_tools(SQUASHFS_TOOLS_DIR, SQUASHFS_TOOLS_URL)
return tools_bin / f"{tool_name}.exe"
resolved = shutil.which(tool_name)
if resolved:
return Path(resolved)
raise FileNotFoundError(
f"{tool_name} not found on PATH. Install squashfs-tools-ng for your Linux distribution "
f"or run the helper on Windows where the bundled MinGW tools are available."
)
def extract_archive(archive: Path, outdir: Path, password: str) -> Path:
ensure_clean_dir(outdir)
with py7zr.SevenZipFile(archive, mode="r", password=password) as zf:
zf.extractall(path=outdir)
top_entries = [entry for entry in outdir.iterdir() if entry.is_dir()]
if len(top_entries) != 1:
raise RuntimeError(f"expected one top-level folder after extraction, found {len(top_entries)}")
return top_entries[0]
def ensure_stock_archive(stock_archive: Path, stock_url: str | None, password: str) -> Path:
if stock_archive.exists():
return stock_archive
if not stock_url:
raise FileNotFoundError(
f"stock archive not found at {stock_archive}; supply --stock-url to download it first"
)
download_file(stock_url, stock_archive)
with py7zr.SevenZipFile(stock_archive, mode="r", password=password) as zf:
zf.test()
return stock_archive
def ensure_squashfs_tools(tools_dir: Path, tools_url: str) -> Path:
bin_dir = tools_dir / "bin"
gensquashfs = bin_dir / "gensquashfs.exe"
rdsquashfs = bin_dir / "rdsquashfs.exe"
if gensquashfs.exists() and rdsquashfs.exists():
return bin_dir
tools_dir.parent.mkdir(parents=True, exist_ok=True)
zip_path = tools_dir.with_suffix(".zip")
if not zip_path.exists():
download_file(tools_url, zip_path)
if tools_dir.exists():
shutil.rmtree(tools_dir)
with zipfile.ZipFile(zip_path) as zf:
zf.extractall(path=tools_dir.parent)
if not gensquashfs.exists() or not rdsquashfs.exists():
raise FileNotFoundError(f"failed to install squashfs tools into {bin_dir}")
return bin_dir
def reconstruct_from_archive_tree(tree_root: Path, workspace: Path) -> dict[str, Path]:
version_file = tree_root / "ota_config.in"
if version_file.exists():
version = read_version(version_file)
ota_dir = tree_root / f"ota_v{version}"
else:
ota_dir = tree_root / f"ota_v{DEFAULT_VERSION}"
if not ota_dir.exists():
matches = sorted([entry for entry in tree_root.iterdir() if entry.is_dir() and entry.name.startswith("ota_v")])
if not matches:
raise FileNotFoundError(f"could not locate ota_v* directory under {tree_root}")
ota_dir = matches[0]
rootfs_full = workspace / "rootfs.squashfs.full"
ximage_full = workspace / "xImage.full"
zero_full = workspace / "zero.bin.full"
rootfs_full.parent.mkdir(parents=True, exist_ok=True)
ximage_full.parent.mkdir(parents=True, exist_ok=True)
zero_full.parent.mkdir(parents=True, exist_ok=True)
reconstruct_split_file(ota_dir, "rootfs.squashfs", rootfs_full)
reconstruct_split_file(ota_dir, "xImage", ximage_full)
reconstruct_split_file(ota_dir, "zero.bin", zero_full)
ximage_payload = workspace / "xImage.payload"
ximage_payload.write_bytes(ximage_full.read_bytes()[UIMAGE_HEADER_SIZE:])
return {
"ota_dir": ota_dir,
"rootfs_full": rootfs_full,
"ximage_full": ximage_full,
"ximage_payload": ximage_payload,
"zero_full": zero_full,
}
def extract_rootfs_to_dir(rootfs_full: Path, outdir: Path) -> None:
ensure_clean_dir(outdir)
rdsquashfs = resolve_squashfs_tool("rdsquashfs")
subprocess.run(
[
str(rdsquashfs),
"--unpack-path",
"/",
"--unpack-root",
str(outdir),
"--chmod",
"--chown",
"--set-times",
"--quiet",
str(rootfs_full),
],
check=True,
)
def describe_rootfs(rootfs_full: Path) -> List[str]:
rdsquashfs = resolve_squashfs_tool("rdsquashfs")
result = subprocess.run(
[str(rdsquashfs), "--describe", str(rootfs_full)],
check=True,
capture_output=True,
text=True,
encoding="utf-8",
errors="replace",
)
return [line.strip() for line in result.stdout.splitlines() if line.strip()]
def describe_rootfs_map(rootfs_full: Path) -> dict[str, str]:
root_entries: dict[str, str] = {}
for line in describe_rootfs(rootfs_full):
parts = shlex.split(line)
if len(parts) < 2:
continue
root_entries[parts[1].lstrip("/")] = line
return root_entries
def packfile_entry(line: str) -> tuple[str, str]:
parts = shlex.split(line)
if len(parts) < 5:
raise ValueError(f"invalid squashfs describe line: {line}")
kind = parts[0]
path = parts[1]
mode = parts[2]
uid = parts[3]
gid = parts[4]
extra = " ".join(parts[5:])
pack_path = path if path.startswith("/") else f"/{path}"
if kind in {"dir", "file"}:
entry = f"{kind} {quote_pack_path(pack_path)} {mode} {uid} {gid}"
elif kind in {"slink", "link"}:
if not extra:
raise ValueError(f"missing link target for {line}")
entry = f"{kind} {quote_pack_path(pack_path)} {mode} {uid} {gid} {extra}"
elif kind == "nod":
if not extra:
raise ValueError(f"missing device metadata for {line}")
entry = f"{kind} {quote_pack_path(pack_path)} {mode} {uid} {gid} {extra}"
else:
raise ValueError(f"unsupported squashfs entry type: {kind}")
return kind, entry
def quote_pack_path(path: str) -> str:
if not any(ch.isspace() for ch in path) and '"' not in path:
return path
return '"' + path.replace("\\", "\\\\").replace('"', '\\"') + '"'
def list_overlay_entries(rootfs_dir: Path, stock_paths: set[str]) -> List[str]:
additions: List[tuple[int, str]] = []
for entry in rootfs_dir.rglob("*"):
relative = entry.relative_to(rootfs_dir).as_posix()
pack_path = f"/{relative}"
if relative in stock_paths:
continue
if entry.is_dir() and not entry.is_symlink():
additions.append((0, f"dir {quote_pack_path(pack_path)} 0755 0 0"))
elif entry.is_symlink():
additions.append((2, f"slink {quote_pack_path(pack_path)} 0777 0 0 {os.readlink(entry)}"))
else:
mode_bits = entry.stat().st_mode & 0o7777
if mode_bits & 0o111 == 0 and relative.startswith("etc/init.d/"):
mode_bits = (mode_bits & ~0o666) | 0o755
mode = oct(mode_bits)[2:]
additions.append((1, f"file {quote_pack_path(pack_path)} {mode} 0 0"))
additions.sort(key=lambda item: (item[0], item[1]))
return [entry for _, entry in additions]
def collect_intentional_overlay_paths(overlay_dir: Path) -> set[str]:
intentional: set[str] = set()
delete_file = overlay_dir / ".delete"
if delete_file.exists():
intentional.update(
str(path).replace("\\", "/")
for path in read_delete_list(overlay_dir)
)
for entry in overlay_dir.rglob("*"):
relative = entry.relative_to(overlay_dir).as_posix()
intentional.add(relative)
return intentional
def should_skip_compare_path(path: str) -> bool:
return any(ord(ch) > 127 for ch in path)
def compare_rootfs_metadata(stock_rootfs_full: Path, rebuilt_rootfs_full: Path, overlay_dir: Path) -> List[str]:
stock_map = describe_rootfs_map(stock_rootfs_full)
rebuilt_map = describe_rootfs_map(rebuilt_rootfs_full)
intentional = collect_intentional_overlay_paths(overlay_dir)
issues: List[str] = []
for path, stock_line in stock_map.items():
if should_skip_compare_path(path):
continue
if path in intentional:
continue
rebuilt_line = rebuilt_map.get(path)
if rebuilt_line is None:
issues.append(f"missing from rebuilt rootfs: {path}")
continue
if rebuilt_line != stock_line:
issues.append(f"metadata mismatch: {path}")
for path, rebuilt_line in rebuilt_map.items():
if should_skip_compare_path(path):
continue
if path in stock_map or path in intentional:
continue
issues.append(f"unexpected extra path in rebuilt rootfs: {path}")
return issues
def write_pack_file(rootfs_dir: Path, stock_rootfs_full: Path, packfile_path: Path) -> None:
lines = describe_rootfs(stock_rootfs_full)
stock_entries: List[str] = []
stock_paths: set[str] = set()
for line in lines:
kind, entry = packfile_entry(line)
path = shlex.split(line)[1]
stock_paths.add(path)
target_path = rootfs_dir / path
if kind in {"file", "dir"}:
if not target_path.exists():
continue
elif kind in {"slink", "link", "nod"}:
if kind in {"slink", "link"} and target_path.exists():
remove_path(target_path)
stock_entries.append(entry)
overlay_entries = list_overlay_entries(rootfs_dir, stock_paths)
packfile_path.write_text("\n".join(stock_entries + overlay_entries) + "\n", encoding="utf-8", newline="\n")
def remove_path(path: Path) -> None:
if not path.exists() and not path.is_symlink():
return
if path.is_dir() and not path.is_symlink():
shutil.rmtree(path)
else:
path.unlink()
def safe_rmtree(path: Path) -> None:
if not path.exists() and not path.is_symlink():
return
if path.is_symlink() or path.is_file():
path.unlink(missing_ok=True)
return
for entry in path.iterdir():
safe_rmtree(entry)
path.rmdir()
def read_delete_list(overlay_dir: Path) -> List[Path]:
delete_file = overlay_dir / ".delete"
if not delete_file.exists():
return []
deletions: List[Path] = []
for raw_line in delete_file.read_text(encoding="utf-8").splitlines():
line = raw_line.strip()
if not line or line.startswith("#"):
continue
deletions.append(Path(line))
return deletions
def apply_overlay_tree(rootfs_dir: Path, overlay_dir: Path) -> None:
if not overlay_dir.exists():
raise FileNotFoundError(f"overlay directory not found: {overlay_dir}")
for relative in read_delete_list(overlay_dir):
target = rootfs_dir / relative
remove_path(target)
for entry in overlay_dir.rglob("*"):
if entry.is_dir():
continue
if entry.name == ".delete" and entry.parent == overlay_dir:
continue
relative = entry.relative_to(overlay_dir)
target = rootfs_dir / relative
target.parent.mkdir(parents=True, exist_ok=True)
remove_path(target)
shutil.copy2(entry, target)
def build_rootfs_from_dir(rootfs_dir: Path, stock_rootfs_full: Path, output_path: Path, workspace: Path) -> None:
gensquashfs = resolve_squashfs_tool("gensquashfs")
packfile_path = workspace / "rootfs.packfile"
write_pack_file(rootfs_dir, stock_rootfs_full, packfile_path)
if output_path.exists():
output_path.unlink()
subprocess.run(
[
str(gensquashfs),
"--pack-file",
str(packfile_path),
"--pack-dir",
str(rootfs_dir),
"--keep-time",
"--block-size",
"131072",
"--compressor",
"gzip",
"--force",
"--quiet",
str(output_path),
],
check=True,
)
def read_version(config_path: Path) -> str:
for line in config_path.read_text(encoding="utf-8").splitlines():
line = line.strip()
if line.startswith("current_version="):
return line.split("=", 1)[1].strip()
raise ValueError(f"current_version not found in {config_path}")
def write_ota_manifest(
staging_root: Path,
version: str,
items: Sequence[Tuple[str, str, Path, Path, List[str]]],
) -> None:
archive_root = staging_root.name
version_dir = staging_root / f"ota_v{version}"
version_dir.mkdir(parents=True, exist_ok=True)
(staging_root / "ota_config.in").write_text(f"current_version={version}\n", encoding="utf-8")
(version_dir / f"ota_v{version}.ok").write_bytes(b"")
ota_update_lines = [f"ota_version={version}", ""]
for img_type, img_name, full_path, md5_path, chunk_md5s in items:
ota_update_lines.extend(
[
f"img_type={img_type}",
f"img_name={img_name}",
f"img_size={full_path.stat().st_size}",
f"img_md5={md5_file(full_path)}",
"",
]
)
sidecar_name = f"ota_md5_{img_name}.{md5_file(full_path)}"
write_lines(version_dir / sidecar_name, chunk_md5s)
write_lines(version_dir / "ota_update.in", ota_update_lines)
def stage_and_pack(
workspace: Path,
version: str,
rootfs_full: Path,
ximage_full: Path,
zero_full: Path,
output_img: Path,
password: str,
) -> Path:
archive_root = f"DarKE_KE_F005_ota_img_V{version}"
staging_root = workspace / "_firmware_stage" / archive_root
ensure_clean_dir(staging_root.parent)
staging_root.mkdir(parents=True, exist_ok=True)
version_dir = staging_root / f"ota_v{version}"
version_dir.mkdir(parents=True, exist_ok=True)
rootfs_md5, rootfs_chunks, rootfs_chunk_md5s = split_file(rootfs_full, version_dir, "rootfs.squashfs")
ximage_md5, ximage_chunks, ximage_chunk_md5s = split_file(ximage_full, version_dir, "xImage")
zero_md5, zero_chunks, zero_chunk_md5s = split_file(zero_full, version_dir, "zero.bin")
(staging_root / "ota_config.in").write_text(f"current_version={version}\n", encoding="utf-8", newline="\n")
(version_dir / f"ota_v{version}.ok").write_bytes(b"")
ota_update_lines = [f"ota_version={version}", ""]
for img_type, img_name, full_path, chunk_md5s, full_md5 in [
("kernel", "xImage", ximage_full, ximage_chunk_md5s, ximage_md5),
("rootfs", "rootfs.squashfs", rootfs_full, rootfs_chunk_md5s, rootfs_md5),
("rtos", "zero.bin", zero_full, zero_chunk_md5s, zero_md5),
]:
ota_update_lines.extend(
[
f"img_type={img_type}",
f"img_name={img_name}",
f"img_size={full_path.stat().st_size}",
f"img_md5={full_md5}",
"",
]
)
sidecar_name = f"ota_md5_{img_name}.{full_md5}"
write_lines(version_dir / sidecar_name, chunk_md5s)
write_lines(version_dir / "ota_update.in", ota_update_lines)
if output_img.exists():
output_img.unlink()
with py7zr.SevenZipFile(output_img, mode="w", password=password) as zf:
zf.writeall(staging_root, arcname=archive_root)
return output_img
def validate_archive(archive: Path, password: str) -> None:
with tempfile.TemporaryDirectory(dir=str(archive.parent)) as tmpdir:
tmp_path = Path(tmpdir)
with py7zr.SevenZipFile(archive, mode="r", password=password) as zf:
zf.extractall(path=tmp_path)
top = next(tmp_path.iterdir(), None)
if top is None or not top.is_dir():
raise RuntimeError("archive validation failed: missing top-level directory")
ota_config = top / "ota_config.in"
if not ota_config.exists():
raise RuntimeError("archive validation failed: missing ota_config.in")
def cmd_extract(args: argparse.Namespace) -> None:
archive = Path(args.archive).expanduser().resolve()
outdir = Path(args.outdir).expanduser().resolve()
workspace = Path(args.workspace).expanduser().resolve()
tree_root = extract_archive(archive, outdir, args.password)
files = reconstruct_from_archive_tree(tree_root, workspace)
extract_rootfs_to_dir(files["rootfs_full"], workspace / "rootfs_extract")
print(f"extracted archive to {outdir}")
print(f"reconstructed rootfs: {files['rootfs_full']}")
print(f"reconstructed kernel: {files['ximage_full']}")
print(f"reconstructed rtos: {files['zero_full']}")
print(f"rootfs unpacked to: {workspace / 'rootfs_extract'}")
def cmd_build(args: argparse.Namespace) -> None:
version = args.version
if not version:
version = DEFAULT_VERSION
build_dir = Path(args.build_dir).expanduser().resolve()
build_dir.mkdir(parents=True, exist_ok=True)
output_img = Path(args.output).expanduser().resolve()
stock_archive = ensure_stock_archive(
Path(args.stock_archive).expanduser().resolve(),
args.stock_url,
args.password,
)
overlay_dir = Path(args.overlay_dir).expanduser().resolve()
workdir = Path(tempfile.mkdtemp(prefix="_firmware_build_", dir=str(build_dir)))
try:
stock_extract_dir = workdir / "stock_extract"
stock_tree = extract_archive(stock_archive, stock_extract_dir, args.password)
stock_files = reconstruct_from_archive_tree(stock_tree, workdir)
rootfs_dir = workdir / "rootfs_work"
extract_rootfs_to_dir(stock_files["rootfs_full"], rootfs_dir)
apply_overlay_tree(rootfs_dir, overlay_dir)
rootfs_full = workdir / "rootfs.squashfs.full"
build_rootfs_from_dir(rootfs_dir, stock_files["rootfs_full"], rootfs_full, workdir)
stage_and_pack(
workspace=workdir,
version=version,
rootfs_full=rootfs_full,
ximage_full=stock_files["ximage_full"],
zero_full=stock_files["zero_full"],
output_img=output_img,
password=args.password,
)
validate_archive(output_img, args.password)
finally:
safe_rmtree(workdir)
print(f"built firmware image: {output_img}")
def cmd_compare(args: argparse.Namespace) -> None:
stock_archive = Path(args.stock_archive).expanduser().resolve()
rebuilt_archive = Path(args.rebuilt_archive).expanduser().resolve()
overlay_dir = Path(args.overlay_dir).expanduser().resolve()
compare_root = Path(args.work_dir).expanduser().resolve()
compare_root.mkdir(parents=True, exist_ok=True)
workdir = Path(tempfile.mkdtemp(prefix="_firmware_compare_", dir=str(compare_root)))
try:
stock_tree = extract_archive(stock_archive, workdir / "stock_extract", args.password)
rebuilt_tree = extract_archive(rebuilt_archive, workdir / "rebuilt_extract", args.password)
stock_files = reconstruct_from_archive_tree(stock_tree, workdir / "stock")
rebuilt_files = reconstruct_from_archive_tree(rebuilt_tree, workdir / "rebuilt")
stock_rootfs_full = stock_files["rootfs_full"]
rebuilt_rootfs_full = rebuilt_files["rootfs_full"]
issues = compare_rootfs_metadata(stock_rootfs_full, rebuilt_rootfs_full, overlay_dir)
if issues:
for issue in issues:
print(issue)
raise SystemExit(1)
finally:
safe_rmtree(workdir)
print("rootfs metadata matches stock outside the overlay")
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description="Extract and rebuild Ender-3 V3 KE firmware images.")
parser.add_argument("--password", default=PASSWORD, help="7z archive password")
subparsers = parser.add_subparsers(dest="command", required=True)
extract = subparsers.add_parser("extract", help="extract an official firmware archive")
extract.add_argument("--archive", required=True, help="path to the stock .img archive")
extract.add_argument("--outdir", default="ke_firmware_extract", help="archive extraction directory")
extract.add_argument("--workspace", default=".", help="workspace for reconstructed artifacts")
extract.set_defaults(func=cmd_extract)
build = subparsers.add_parser("build", help="build a repacked firmware archive")
build.add_argument("--workspace", default=".", help="workspace root")
build.add_argument("--build-dir", default="build", help="directory for intermediate build artifacts")
build.add_argument(
"--stock-archive",
default=str(DEFAULT_STOCK_ARCHIVE),
help="path to the stock OTA image to use as the build base",
)
build.add_argument(
"--stock-url",
default=None,
help="download URL used when the stock OTA image is missing",
)
build.add_argument(
"--overlay-dir",
default=str(DEFAULT_OVERLAY_DIR),
help="directory containing only our custom rootfs changes",
)
build.add_argument(
"--output",
default=str(Path("build") / f"DarKE_KE_F005_ota_img_V{DEFAULT_VERSION}.img"),
help="output .img archive path",
)
build.add_argument("--version", default=DEFAULT_VERSION, help="firmware version to encode in manifests")
build.set_defaults(func=cmd_build)
compare = subparsers.add_parser("compare", help="compare a rebuilt image's rootfs metadata against stock")
compare.add_argument("--stock-archive", required=True, help="path to the stock .img archive")
compare.add_argument("--rebuilt-archive", required=True, help="path to the rebuilt .img archive")
compare.add_argument(
"--overlay-dir",
default=str(DEFAULT_OVERLAY_DIR),
help="overlay tree used to produce the rebuilt image",
)
compare.add_argument("--work-dir", default="build", help="directory for temporary compare artifacts")
compare.set_defaults(func=cmd_compare)
return parser
def main(argv: Sequence[str] | None = None) -> int:
for stream in (sys.stdout, sys.stderr):
try:
stream.reconfigure(encoding="utf-8", errors="replace")
except Exception:
pass
parser = build_parser()
args = parser.parse_args(argv)
args.func(args)
return 0
if __name__ == "__main__":
raise SystemExit(main())