0.3.0更新

提高仿真深度
Vivado 版本查找器。
This commit is contained in:
3A1 2025-06-26 00:30:28 +08:00
parent 0e6aa4002d
commit 21c84754c9
23 changed files with 1677 additions and 212 deletions

View File

@ -125,7 +125,6 @@ secrets/
.secrets/
flash_fpga.py
requirements-test.txt
.github/
.pytest_cache/
.benchmarks/
.benchmarks/

8
.gitignore vendored
View File

@ -376,3 +376,11 @@ secrets.json
*.patch
*.diff
bandit-report.json
junit-external.xml
junit-unit.xml
external_sv_example.sv
external_tcl_example.tcl
junit-unit.xml
test.tcl
junit-unit.xml
generate.log

View File

@ -1,57 +1,171 @@
# Changelog
# 📝 Changelog
[![PyPI version](https://badge.fury.io/py/pcileech-fw-generator.svg)](https://badge.fury.io/py/pcileech-fw-generator)
[![Python Support](https://img.shields.io/pypi/pyversions/pcileech-fw-generator.svg)](https://pypi.org/project/pcileech-fw-generator/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
All notable changes to the PCILeech Firmware Generator will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
and this project adheres to [Semantic Versioning](https://semver.org/spec/v0.1.2.html).
## [2.0.0] - 2025-01-02
---
### Added
- **Interactive TUI Interface**: Complete text-based user interface with real-time monitoring
- Visual PCIe device browser with enhanced device information
- Guided configuration workflows with validation
- Real-time build monitoring with progress tracking
- System status monitoring for Podman, Vivado, USB devices
- Intelligent error guidance with suggested fixes
- Profile management for build configurations
- **Enhanced Package Structure**: Professional Python packaging with pip installability
- Console script entry points (`pcileech-generate`, `pcileech-tui`, `pcileech-build`)
- Proper package metadata and dependency management
- Optional TUI dependencies for lightweight installations
- **Advanced SystemVerilog Features**: Comprehensive PCIe device controller improvements
- Modular architecture with enhanced power management
- Performance counters and monitoring capabilities
- Error handling and recovery mechanisms
- Manufacturing variance simulation for realistic behavior
- **Behavioral Profiling**: Dynamic device behavior capture and simulation
- Real-time register access pattern analysis
- Timing characteristic profiling
- Device-specific behavior modeling
- **Quality Assurance**: Comprehensive testing and code quality tools
- Unit and integration test suites
- Code formatting with Black and isort
- Type checking with mypy
- Pre-commit hooks for development workflow
- **Container Improvements**: Enhanced containerized build environment
- Updated Containerfile with TUI support
- Improved resource management and monitoring
- Better error handling and logging
## 📑 Table of Contents
### Changed
- **Major Version Bump**: Incremented to v2.0.0 to reflect significant TUI addition
- **Improved Documentation**: Enhanced README with TUI features and installation instructions
- **Better Error Handling**: More informative error messages and recovery suggestions
- **Enhanced Logging**: Improved logging throughout the application with structured output
- [Version 0.3.0 (2025-06-09)](#020---2025-06-09)
- [Version 0.3.0 (2025-01-02)](#0110---2025-01-02)
- [Release Notes](#release-notes)
- [Backward Compatibility](#backward-compatibility)
- [Future Roadmap](#future-roadmap)
### Technical Details
- **Dependencies**: Added Textual, Rich, Watchdog for TUI functionality
- **Python Support**: Requires Python 3.9+ with support through 3.12
- **Package Structure**: Reorganized as proper Python package with setuptools/pip support
- **Entry Points**: Added console scripts for easy command-line access
- **Testing**: Comprehensive test suite with pytest and coverage reporting
---
### Installation
## [0.3.0] - 2025-06-10
### ✨ Added
- **🧩 Feature Integration**: Comprehensive integration of all major features
- Integrated documentation in `docs/INTEGRATED_FEATURES.md`
- Comprehensive integration tests in `tests/test_feature_integration.py`
- Seamless interoperation between all components
- **💾 Full 4 KB Config-Space Shadow in BRAM**: Complete configuration space emulation
- Full 4 KB configuration space shadow in BRAM
- Dual-port access for simultaneous read/write operations
- Overlay RAM for writable fields (Command/Status registers)
- Initialization from donor device configuration data
- Little-endian format compatible with PCIe specification
- **🔄 Auto-Replicate MSI-X Table**: Exact MSI-X table replication
- Automatic parsing of MSI-X capability structure from donor configuration space
- Parameterized SystemVerilog implementation of MSI-X table and PBA
- Support for byte-enable granularity writes
- Interrupt delivery logic with masking support
- Integration with the existing BAR controller and configuration space shadow
- **✂️ Capability Pruning**: Selective capability modification
- Automatic analysis of standard and extended PCI capabilities
- Categorization of capabilities based on emulation feasibility
- Selective pruning of unsupported capabilities
- Modification of partially supported capabilities
- Preservation of capability chain integrity
- **🎲 Deterministic Variance Seeding**: Consistent hardware variance
- Deterministic seed generation based on device serial number (DSN) and build revision
- Consistent variance parameters for the same donor device and build revision
- Different variance parameters for different donor devices or build revisions
- Support for different device classes with appropriate variance ranges
- **🏗️ Build Process**: Enhanced to support all integrated features
- Improved donor dump extraction with full 4 KB configuration space
- Added capability pruning step to the build process
- Added MSI-X table parameter extraction and integration
- Added deterministic variance seeding based on DSN and build revision
- **📋 Enhanced Logging**: Improved logging for all integrated features
- Added detailed logging for capability pruning
- Added MSI-X table parameter logging
- Added variance parameter logging
- Added integration status summary at the end of the build process
- **📚 Documentation**: Comprehensive documentation for all integrated features
- Added `docs/INTEGRATED_FEATURES.md` with detailed integration information
- Updated feature-specific documentation with integration details
- Added troubleshooting information for integrated features
- **🔌 MSI-X Table Integration**: Fixed issues with MSI-X table integration
- Corrected MSI-X table parameter extraction from configuration space
- Fixed MSI-X table and PBA memory mapping in BAR controller
- Improved error handling for MSI-X capability parsing
- **🧩 Capability Chain Integrity**: Fixed issues with capability chain integrity
- Ensured proper next pointer updates when removing capabilities
- Fixed extended capability chain traversal and modification
- Improved error handling for capability chain manipulation
- **⏱️ Timing Consistency**: Fixed issues with timing consistency
- Ensured deterministic variance seeding produces consistent results
- Fixed timing parameter calculation and application
- Improved error handling for variance parameter generation
## [0.3.0] - 2025-06-09
### ✨ Added
- **💾 Option-ROM Passthrough**: Complete Option-ROM replication from donor devices
- Extracts Option-ROM from donor PCI devices using Linux sysfs interface
- Supports two implementation modes:
- Mode A: BAR 5 Window (pure FPGA implementation)
- Mode B: External SPI Flash (for larger ROMs)
- Handles legacy 16-bit config cycles for ROM access
- Includes caching for improved performance
- Configurable ROM size and source
- **🔧 Build System Integration**: Enhanced build process for Option-ROM support
- Added command-line arguments for enabling and configuring Option-ROM feature
- Automatic ROM extraction during build process
- Support for using pre-extracted ROM files
- Build-time selection between implementation modes
- **🧪 Testing Infrastructure**: Comprehensive test suite for Option-ROM functionality
- Unit tests for Option-ROM extraction and handling
- Support for different ROM sizes and formats
- Validation of ROM signature and content
### 🔄 Changed
- **🔢 Version Bump**: Incremented to v0.3.0 to reflect significant Option-ROM feature addition
- **🏗️ Build Process**: Updated to support Option-ROM integration
- **📋 Enhanced Logging**: Improved logging for Option-ROM extraction and processing
## [Unreleased] - Build Process Improvements
### 🔄 Changed
- **🏗️ Local Build Default**: Changed local build to be the default process
- Local builds now run by default (no container required)
- Container builds now require explicit opt-in with `--use-donor-dump`
- Improved error handling for local build scenarios
- Enhanced documentation for local build workflows
- **🔧 Container Engine Options**: Added support for multiple container engines
- Added new `--container-engine` option to specify engine preference
- Podman is now the default container engine
- Docker remains fully supported as an alternative option
- Automatic detection of available container engines
- **🔍 Vivado Location Validation**: Enhanced Vivado detection and validation
- Improved cross-platform Vivado installation detection
- Added support for environment variables (XILINX_VIVADO)
- Automatic version detection and compatibility checking
- Detailed error messages for missing or incompatible installations
### 🔧 Fixed
- **🔌 VFIO Device Binding**: Fixed an issue where binding a device already bound to vfio-pci would fail
- Added detection for devices already bound to vfio-pci
- Improved error handling during the binding process
- Added comprehensive test cases for this edge case
- **📦 Container Dependency Installation**: Fixed missing Python dependencies in container build
- Added proper `pip install` commands for `requirements.txt` and `requirements-tui.txt`
- Fixed import errors for `psutil`, `pydantic`, and other required packages
- **📁 Container File Structure**: Corrected file paths and directory structure
- Fixed `build.py` path from `/app/build.py` to `/app/src/build.py`
- Updated all container usage examples and documentation
- **🔒 Container Security Improvements**: Enhanced security posture
- Replaced `--privileged` with specific capabilities (`--cap-add=SYS_RAWIO --cap-add=SYS_ADMIN`)
- Maintained non-root user execution while preserving functionality
- **✅ Container Health Checks**: Improved dependency validation
- Enhanced health check to validate Python imports
- Added comprehensive dependency testing
### ✨ Added
- **🔨 Container Build Script**: New automated build and test script
- Added `scripts/build_container.sh` with comprehensive testing
- Supports both Podman and Docker container engines
- Includes security validation and usage examples
- **🚀 Container CI Pipeline**: Automated container testing workflow
- Added `.github/workflows/container-ci.yml` for continuous integration
- Tests container build, dependencies, security, and integration
- Validates file structure and user permissions
### 📚 Improved
- **📖 Documentation Updates**: Enhanced container usage documentation
- Updated `podman_demo.md` with security best practices
- Added troubleshooting section for container issues
- Included capability-based security examples
### 🗂️ Changed
- **📦 Container File Inclusion**: Updated `.dockerignore` configuration
- Removed exclusion of `src/tui/` components
- Included necessary requirements files
- Optimized build context for better performance
---
### 🚀 Installation
```bash
# Basic installation
pip install pcileech-fw-generator
@ -63,7 +177,7 @@ pip install pcileech-fw-generator[tui]
pip install pcileech-fw-generator[dev]
```
### Usage
### 🎮 Usage
```bash
# Command line interface (traditional)
pcileech-generate
@ -77,7 +191,7 @@ pcileech-build --bdf 0000:03:00.0 --board 75t
## [1.0.0] - 2024-12-01
### Added
### Added
- Initial release of PCILeech Firmware Generator
- Basic command-line interface for firmware generation
- Donor hardware analysis and configuration extraction
@ -86,7 +200,7 @@ pcileech-build --bdf 0000:03:00.0 --board 75t
- Basic SystemVerilog generation for PCIe devices
- Podman-based isolated build environment
### Features
### 🎯 Features
- PCIe device enumeration and selection
- Configuration space extraction from donor hardware
- FPGA bitstream generation for Artix-7 boards
@ -95,26 +209,54 @@ pcileech-build --bdf 0000:03:00.0 --board 75t
---
## Release Notes
## 📋 Release Notes
### v2.0.0 Highlights
### 🚀 v0.3.0 Highlights
This release integrates all major features of the PCILeech FPGA firmware generator, providing a comprehensive solution for PCIe device emulation. The integration ensures that all features work together seamlessly, providing a more realistic and functional emulation experience.
Key improvements include:
- **💾 Full 4 KB Config-Space Shadow**: Complete configuration space emulation with overlay RAM for writable fields
- **🔄 MSI-X Table Replication**: Exact replication of MSI-X tables from donor devices
- **✂️ Capability Pruning**: Selective modification of capabilities that can't be faithfully emulated
- **🎲 Deterministic Variance Seeding**: Consistent hardware variance based on device serial number and build revision
### 🚀 v0.3.0 Highlights
This release introduces the Option-ROM passthrough feature, allowing the PCILeech FPGA firmware to faithfully replicate the Option-ROM of donor PCI devices. This enables advanced functionality such as UEFI boot support and device-specific initialization.
Key improvements include:
- **💾 Complete Option-ROM Replication**: Extract and replicate Option-ROMs from donor devices
- **🔀 Dual Implementation Modes**: Choose between pure FPGA (BAR window) or SPI flash implementations
- **🔌 Legacy ROM Support**: Proper handling of legacy 16-bit config cycles for ROM access
- **🛠️ Flexible Configuration**: Command-line options for ROM source, size, and implementation mode
### 🚀 v0.3.0 Highlights
This major release introduces a modern, interactive TUI that transforms the user experience while maintaining full backward compatibility with the original command-line interface. The TUI provides guided workflows, real-time monitoring, and intelligent error handling that makes firmware generation more accessible and reliable.
Key improvements include:
- **Zero Learning Curve**: Intuitive interface guides users through the entire process
- **Real-time Feedback**: Live monitoring of build progress and system resources
- **Error Prevention**: Validation and checks prevent common configuration mistakes
- **Professional Packaging**: Easy installation via pip with proper dependency management
- **🎯 Zero Learning Curve**: Intuitive interface guides users through the entire process
- **📊 Real-time Feedback**: Live monitoring of build progress and system resources
- **🛡️ Error Prevention**: Validation and checks prevent common configuration mistakes
- **📦 Professional Packaging**: Easy installation via pip with proper dependency management
### Backward Compatibility
### 🔄 Backward Compatibility
All existing command-line workflows continue to work unchanged. The TUI is an optional enhancement that requires additional dependencies, ensuring lightweight installations remain possible.
All existing command-line workflows continue to work unchanged. The new integrated features are designed to be backward compatible with existing workflows, ensuring a smooth transition for users.
### Future Roadmap
### 🔮 Future Roadmap
- Web-based interface for remote build management
- Enhanced device compatibility and detection
- Advanced firmware customization options
- Integration with additional FPGA toolchains
- Cloud-based build services
- Cloud-based build services
## ⚠️ Disclaimer
This tool is intended for educational research and legitimate PCIe development purposes only. Users are responsible for ensuring compliance with all applicable laws and regulations. The authors assume no liability for misuse of this software.
---
**Version 0.3.0** - Major release with integrated features for comprehensive PCIe device emulation

View File

@ -1,23 +1,60 @@
# Contributing to PCILeech Firmware Generator
# 🤝 Contributing to PCILeech Firmware Generator
[![PyPI version](https://badge.fury.io/py/pcileech-fw-generator.svg)](https://badge.fury.io/py/pcileech-fw-generator)
[![Python Support](https://img.shields.io/pypi/pyversions/pcileech-fw-generator.svg)](https://pypi.org/project/pcileech-fw-generator/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![CI](https://github.com/ramseymcgrath/PCILeechFWGenerator/workflows/CI/badge.svg)](https://github.com/ramseymcgrath/PCILeechFWGenerator/actions)
Thank you for your interest in contributing to the PCILeech Firmware Generator! This document provides guidelines and information for contributors.
## Table of Contents
---
- [Code of Conduct](#code-of-conduct)
- [Getting Started](#getting-started)
- [Development Setup](#development-setup)
- [Contributing Guidelines](#contributing-guidelines)
- [Testing](#testing)
- [Code Style](#code-style)
- [Submitting Changes](#submitting-changes)
- [Release Process](#release-process)
## 📑 Table of Contents
## Code of Conduct
- [📜 Code of Conduct](#-code-of-conduct)
- [🚀 Getting Started](#-getting-started)
- [Prerequisites](#prerequisites)
- [Development Setup](#development-setup)
- [📋 Contributing Guidelines](#-contributing-guidelines)
- [Types of Contributions](#types-of-contributions)
- [Bug Reports](#bug-reports)
- [Feature Requests](#feature-requests)
- [🔄 Development Workflow](#-development-workflow)
- [Branch Strategy](#branch-strategy)
- [Making Changes](#making-changes)
- [Commit Message Format](#commit-message-format)
- [🧪 Testing](#-testing)
- [Test Structure](#test-structure)
- [Writing Tests](#writing-tests)
- [Running Tests](#running-tests)
- [💻 Code Style](#-code-style)
- [Python Style Guide](#python-style-guide)
- [Formatting Tools](#formatting-tools)
- [Pre-commit Hooks](#pre-commit-hooks)
- [📚 Documentation](#-documentation)
- [Code Documentation](#code-documentation)
- [Documentation Style](#documentation-style)
- [📤 Submitting Changes](#-submitting-changes)
- [Pull Request Process](#pull-request-process)
- [Pull Request Template](#pull-request-template)
- [Review Process](#review-process)
- [📦 Release Process](#-release-process)
- [Version Management](#version-management)
- [Release Steps](#release-steps)
- [Distribution](#distribution)
- [❓ Getting Help](#-getting-help)
- [Communication Channels](#communication-channels)
- [Development Resources](#development-resources)
- [🏆 Recognition](#-recognition)
- [⚠️ Disclaimer](#-disclaimer)
---
## 📜 Code of Conduct
This project adheres to a code of conduct that promotes a welcoming and inclusive environment. By participating, you are expected to uphold this code.
## Getting Started
## 🚀 Getting Started
### Prerequisites
@ -56,7 +93,7 @@ This project adheres to a code of conduct that promotes a welcoming and inclusiv
python -m pytest tests/
```
## Contributing Guidelines
## 📋 Contributing Guidelines
### Types of Contributions
@ -87,7 +124,7 @@ For feature requests, please provide:
- **Alternatives Considered**: Other approaches you've thought about
- **Implementation Ideas**: Technical approach if you have one
## Development Workflow
## 🔄 Development Workflow
### Branch Strategy
@ -156,7 +193,7 @@ docs(readme): update installation instructions for TUI
test(core): add unit tests for device manager
```
## Testing
## 🧪 Testing
### Test Structure
@ -194,7 +231,7 @@ python -m pytest -n auto
python -m pytest -m "not slow"
```
## Code Style
## 💻 Code Style
### Python Style Guide
@ -230,7 +267,7 @@ Our pre-commit configuration automatically runs:
- mypy (type checking)
- pytest (basic tests)
## Documentation
## 📚 Documentation
### Code Documentation
@ -262,7 +299,7 @@ def generate_firmware(device_bdf: str, board_type: str) -> Path:
"""
```
## Submitting Changes
## 📤 Submitting Changes
### Pull Request Process
@ -304,7 +341,7 @@ Brief description of changes
4. **Approval**: At least one maintainer approval required
5. **Merge**: Changes merged to appropriate branch
## Release Process
## 📦 Release Process
### Version Management
@ -338,7 +375,7 @@ twine check dist/*
twine upload dist/*
```
## Getting Help
## Getting Help
### Communication Channels
@ -353,11 +390,19 @@ twine upload dist/*
- **Examples**: See `examples/` directory
- **Test Cases**: See `tests/` directory
## Recognition
## 🏆 Recognition
Contributors are recognized in:
- **CHANGELOG.md**: Release notes mention contributors
- **GitHub Contributors**: Automatic recognition
- **Release Notes**: Major contributions highlighted
Thank you for contributing to PCILeech Firmware Generator!
## ⚠️ Disclaimer
This tool is intended for educational research and legitimate PCIe development purposes only. Users are responsible for ensuring compliance with all applicable laws and regulations. The authors assume no liability for misuse of this software.
---
Thank you for contributing to PCILeech Firmware Generator!
**Version 0.3.0** - Major release with TUI interface and professional packaging

View File

@ -1,6 +1,6 @@
# Multi-stage build for smaller final image
# Build stage - includes development tools
FROM ubuntu:22.04@sha256:77906da86b60585ce12215807090eb327e7386c8fafb5402369e421f44eff17e AS builder
FROM ubuntu:22.04 AS builder
# Set environment variables for reproducible builds
ENV DEBIAN_FRONTEND=noninteractive \
@ -8,15 +8,14 @@ ENV DEBIAN_FRONTEND=noninteractive \
LANG=C.UTF-8 \
LC_ALL=C.UTF-8
# Install build dependencies with pinned versions
# Install build dependencies without version pins for compatibility
RUN apt-get update && apt-get install -y \
build-essential=12.9ubuntu3 \
linux-headers-generic=5.15.0.91.88 \
make=4.3-4.1build1 \
git=1:2.34.1-1ubuntu1.10 \
python3=3.10.6-1~22.04 \
python3-pip=22.0.2+dfsg-1ubuntu0.4 \
python3-dev=3.10.6-1~22.04 \
build-essential \
make \
git \
python3 \
python3-pip \
python3-dev \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
@ -26,11 +25,12 @@ COPY ./generate.py /build/
COPY ./requirements-test.txt /build/
WORKDIR /build
# Pre-compile any Python modules or build kernel modules if needed
RUN cd src/donor_dump && make clean && make
# Note: Kernel module (donor_dump) should be built on target system, not in container
# The module requires kernel headers matching the host kernel version
# Build instructions are available in src/donor_dump/Makefile
# Runtime stage - minimal runtime environment
FROM ubuntu:22.04@sha256:77906da86b60585ce12215807090eb327e7386c8fafb5402369e421f44eff17e AS runtime
FROM ubuntu:22.04 AS runtime
# Set environment variables for reproducible builds
ENV DEBIAN_FRONTEND=noninteractive \
@ -38,15 +38,15 @@ ENV DEBIAN_FRONTEND=noninteractive \
LANG=C.UTF-8 \
LC_ALL=C.UTF-8
# Install only runtime dependencies with pinned versions
# Install only runtime dependencies without version pins for compatibility
RUN apt-get update && apt-get install -y \
python3=3.10.6-1~22.04 \
python3-pip=22.0.2+dfsg-1ubuntu0.4 \
pciutils=1:3.7.0-6 \
bsdextrautils=2.37.2-4ubuntu3 \
sudo=1.9.9-1ubuntu2.4 \
kmod=29-1ubuntu1 \
ca-certificates=20230311ubuntu0.22.04.1 \
python3 \
python3-pip \
pciutils \
bsdextrautils \
sudo \
kmod \
ca-certificates \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* \
&& rm -rf /tmp/* \
@ -58,80 +58,41 @@ RUN groupadd -r appuser && useradd -r -g appuser -s /bin/bash -m appuser
# Set working directory
WORKDIR /app
# Copy Python requirements first for better layer caching
COPY --chown=appuser:appuser ./requirements.txt /app/
COPY --chown=appuser:appuser ./requirements-tui.txt /app/
COPY --chown=appuser:appuser ./requirements-test.txt /app/
# Install Python dependencies
RUN pip3 install --no-cache-dir --upgrade pip && \
pip3 install --no-cache-dir -r /app/requirements.txt && \
pip3 install --no-cache-dir -r /app/requirements-tui.txt
# Copy application files from builder stage
# This includes all new advanced SystemVerilog modules:
# - manufacturing_variance.py, advanced_sv_*.py, enhanced behavior_profiler.py, enhanced build.py
COPY --from=builder --chown=appuser:appuser /build/src /app
COPY --from=builder --chown=appuser:appuser /build/src /app/src
# Copy additional root-level files needed for enhanced functionality
COPY --chown=appuser:appuser ./generate.py /app/
COPY --chown=appuser:appuser ./requirements-test.txt /app/
# Create output directory with proper permissions
RUN mkdir -p /app/output && chown appuser:appuser /app/output
# Add health check
# Add health check with dependency validation
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD python3 -c "import sys; sys.exit(0)" || exit 1
CMD python3 -c "import psutil, pydantic; print('Dependencies OK')" || exit 1
# Add container usage documentation
LABEL maintainer="Ramsey McGrath <ramseymcgrath@gmail.com>" \
LABEL maintainer="Ramsey McGrath <ramsey@voltcyclone.info>" \
description="PCILeech DMA firmware generator container with advanced SystemVerilog features (multi-stage optimized)" \
version="2.0" \
usage="podman run --rm -it --privileged --device=/dev/vfio/X --device=/dev/vfio/vfio -v ./output:/app/output dma-fw sudo python3 /app/build.py --bdf XXXX:XX:XX.X --board XXt [--advanced-sv] [--device-type TYPE] [--enable-variance]" \
usage="podman run --rm -it --cap-add=SYS_RAWIO --cap-add=SYS_ADMIN --device=/dev/vfio/X --device=/dev/vfio/vfio -v ./output:/app/output dma-fw sudo python3 /app/src/build.py --bdf XXXX:XX:XX.X --board XXt [--advanced-sv] [--device-type TYPE] [--enable-variance]" \
security.notes="Requires privileged mode for PCI device access via VFIO" \
features="Basic firmware generation, Advanced SystemVerilog features, Manufacturing variance simulation, Device-specific optimizations"
# Create entrypoint script for better container usage
RUN echo '#!/bin/bash\n\
set -e\n\
\n\
# Display usage information\n\
if [ "$1" = "--help" ] || [ "$1" = "-h" ]; then\n\
echo "PCILeech DMA Firmware Generator Container v2.0"\n\
echo "Usage: podman run --rm -it --privileged \\"\n\
echo " --device=/dev/vfio/GROUP --device=/dev/vfio/vfio \\"\n\
echo " -v ./output:/app/output dma-fw \\"\n\
echo " sudo python3 /app/build.py [OPTIONS]"\n\
echo ""\n\
echo "Required arguments:"\n\
echo " --bdf XXXX:XX:XX.X PCI Bus:Device.Function (e.g., 0000:03:00.0)"\n\
echo " --board XXt Target board (35t, 75t, or 100t)"\n\
echo ""\n\
echo "Advanced SystemVerilog options:"\n\
echo " --advanced-sv Enable advanced SystemVerilog generation"\n\
echo " --device-type TYPE Device type optimization (network, storage, graphics)"\n\
echo " --enable-variance Enable manufacturing variance simulation"\n\
echo " --disable-power-management Disable power management features"\n\
echo " --enable-debug Enable debug features in generated code"\n\
echo " --custom-config FILE Use custom configuration file"\n\
echo ""\n\
echo "Basic examples:"\n\
echo " # Standard firmware generation"\n\
echo " sudo python3 /app/build.py --bdf 0000:03:00.0 --board 75t"\n\
echo ""\n\
echo "Advanced examples:"\n\
echo " # Basic advanced generation"\n\
echo " sudo python3 /app/build.py --bdf 0000:03:00.0 --board 75t --advanced-sv"\n\
echo ""\n\
echo " # Device-specific with variance"\n\
echo " sudo python3 /app/build.py --bdf 0000:03:00.0 --board 75t --advanced-sv --device-type network --enable-variance"\n\
echo ""\n\
echo " # Custom configuration"\n\
echo " sudo python3 /app/build.py --bdf 0000:03:00.0 --board 75t --advanced-sv --device-type storage --disable-power-management"\n\
echo ""\n\
echo "Features:"\n\
echo " - Basic PCILeech DMA firmware generation"\n\
echo " - Advanced SystemVerilog code generation"\n\
echo " - Manufacturing variance simulation"\n\
echo " - Device-specific optimizations"\n\
echo " - Power management controls"\n\
echo " - Performance profiling and optimization"\n\
exit 0\n\
fi\n\
\n\
# Execute the command\n\
exec "$@"\n' > /entrypoint.sh && chmod +x /entrypoint.sh
# Copy the pre-created entrypoint script
COPY entrypoint.sh /entrypoint.sh
# Note: Vivado tools should be installed separately or use Xilinx's official container images
# This container provides the optimized base environment for DMA firmware generation

View File

@ -24,8 +24,9 @@ recursive-include src/tui/styles *.tcss
# Include donor dump C code and Makefile
recursive-include src/donor_dump *.c *.h Makefile
# Include test files
recursive-include tests *.py
# Exclude test files from distribution
# Tests are available in the source repository but not in PyPI packages
# recursive-include tests *.py
# Include scripts
recursive-include src/scripts *.py

129
Makefile Normal file
View File

@ -0,0 +1,129 @@
# Makefile for PCILeech Firmware Generator
.PHONY: help clean install install-dev test lint format build build-pypi upload-test upload-pypi release
# Default target
help:
@echo "PCILeech Firmware Generator - Available targets:"
@echo ""
@echo "Development:"
@echo " install - Install package in development mode"
@echo " install-dev - Install development dependencies"
@echo " test - Run test suite"
@echo " lint - Run code linting"
@echo " format - Format code with black and isort"
@echo " clean - Clean build artifacts"
@echo ""
@echo "Building:"
@echo " build - Build package distributions"
@echo " build-pypi - Full PyPI package generation (recommended)"
@echo " build-quick - Quick build without quality checks"
@echo ""
@echo "Publishing:"
@echo " upload-test - Upload to Test PyPI"
@echo " upload-pypi - Upload to PyPI"
@echo " release - Full release process"
@echo ""
@echo "Utilities:"
@echo " check-deps - Check system dependencies"
@echo " security - Run security scans"
# Development targets
install:
pip install -e .
install-dev:
pip install -e ".[dev,test,tui]"
test:
pytest tests/ --cov=src --cov-report=term-missing
lint:
flake8 src/ tests/
mypy src/
format:
black src/ tests/
isort src/ tests/
clean:
rm -rf build/ dist/ *.egg-info/ .pytest_cache/ htmlcov/ .coverage
find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
find . -type f -name "*.pyc" -delete 2>/dev/null || true
# Building targets
build:
python -m build
build-pypi:
@echo "Running full PyPI package generation..."
python3 scripts/generate_pypi_package.py --skip-upload
build-quick:
@echo "Running quick PyPI package generation..."
python3 scripts/generate_pypi_package.py --quick --skip-upload
# Publishing targets
upload-test:
@echo "Building and uploading to Test PyPI..."
python3 scripts/generate_pypi_package.py --test-pypi
upload-pypi:
@echo "Building and uploading to PyPI..."
python3 scripts/generate_pypi_package.py
release:
@echo "Running full release process..."
./scripts/build_release.sh release $(VERSION)
# Utility targets
check-deps:
@echo "Checking system dependencies..."
@python3 scripts/generate_pypi_package.py --skip-quality --skip-security --skip-upload --skip-install-test || true
security:
@echo "Running security scans..."
bandit -r src/
safety check
# Docker targets
docker-build:
./scripts/build_container.sh
# Test package build
test-build:
@echo "Testing PyPI package build..."
python3 scripts/test_package_build.py
# Help for specific targets
help-build:
@echo "Build targets:"
@echo ""
@echo " build - Basic build using python -m build"
@echo " build-pypi - Full PyPI generation with all checks"
@echo " build-quick - Quick build skipping quality checks"
@echo ""
@echo "Options for build-pypi:"
@echo " - Runs code quality checks (black, isort, flake8, mypy)"
@echo " - Runs security scans (bandit, safety)"
@echo " - Runs test suite with coverage"
@echo " - Validates package structure"
@echo " - Tests installation in virtual environment"
@echo ""
@echo "Use 'make build-quick' for faster iteration during development"
help-upload:
@echo "Upload targets:"
@echo ""
@echo " upload-test - Upload to Test PyPI (https://test.pypi.org/)"
@echo " upload-pypi - Upload to production PyPI (https://pypi.org/)"
@echo ""
@echo "Prerequisites:"
@echo " - Configure ~/.pypirc with your API tokens"
@echo " - Or set TWINE_USERNAME and TWINE_PASSWORD environment variables"
@echo ""
@echo "Test PyPI installation:"
@echo " pip install --index-url https://test.pypi.org/simple/ pcileech-fw-generator"
@echo ""
@echo "Production PyPI installation:"
@echo " pip install pcileech-fw-generator"

687
README.md
View File

@ -1,35 +1,662 @@
# PCILeech 固件生成器
[![许可证: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
# PCILeech Firmware Generator
通过单条命令从真实捐赠硬件生成仿冒PCIe DMA固件。完整工作流程包括提取捐赠设备配置空间、在隔离容器中构建个性化FPGA比特流、以及可选通过USB-JTAG刷写DMA板卡。
[![PyPI - Version](https://img.shields.io/pypi/v/:pcileechfwgenerator)](https://pypi.org/project/pcileechfwgenerator/)
[![CI](https://github.com/ramseymcgrath/PCILeechFWGenerator/workflows/CI/badge.svg)](https://github.com/ramseymcgrath/PCILeechFWGenerator/actions)
[![codecov](https://codecov.io/gh/ramseymcgrath/PCILeechFWGenerator/branch/main/graph/badge.svg)](https://codecov.io/gh/ramseymcgrath/PCILeechFWGenerator)
![](https://dcbadge.limes.pink/api/shield/429866199833247744)
## ✨ 核心特性
Generate spoofed PCIe DMA firmware from real donor hardware with a single command. The workflow rips the donor's configuration space, builds a personalized FPGA bitstream locally by default (or optionally in an isolated container), and (optionally) flashes your DMA card over USBJTAG.
### 硬件分析
- **🎯 捐赠硬件分析**
完整提取PCIe设备的配置空间和寄存器映射
- **📊 行为分析**
动态捕获设备操作模式默认采样10秒
---
### 仿真增强
- **🔧 制造差异模拟**
支持四类设备仿真:
- `消费级`2-5%时钟抖动
- `企业级`1-3%参数变异
- `工业级`:扩展温度范围
- `车规级`:增强稳定性
## 📑 Table of Contents
### 代码生成
- **⚡ 高级SystemVerilog生成**
模块化架构包含:
```systemverilog
// 示例PCIe电源管理模块
module pcie_power_management (
input wire clk,
input wire [1:0] power_state,
output reg l1_ready
);
/* 支持D0-D3cold全状态 */
endmodule
- [✨ Features](#-features)
- [🚀 Quick Start](#-quick-start)
- [Installation](#installation)
- [Usage](#usage)
- [📋 Requirements](#-requirements)
- [Software](#software)
- [Hardware](#hardware)
- [🛠️ Installation & Setup](#-installation--setup)
- [🎮 Usage](#-usage-1)
- [Interactive TUI Mode](#interactive-tui-mode-recommended)
- [Command Line Mode](#command-line-mode)
- [Legacy Mode](#legacy-mode-source-installation)
- [🔌 Flashing the DMA Board](#-flashing-the-dma-board)
- [🚀 Advanced Features](#-advanced-features)
- [Full 4 KB Config-Space Shadow](#full-4-kb-config-space-shadow)
- [MSI-X Table Replication](#msi-x-table-replication)
- [Capability Pruning](#capability-pruning)
- [Deterministic Variance Seeding](#deterministic-variance-seeding)
- [Manufacturing Variance Simulation](#manufacturing-variance-simulation)
- [Advanced SystemVerilog Generation](#advanced-systemverilog-generation)
- [Behavioral Profiling](#behavioral-profiling)
- [Command-Line Options](#command-line-options)
- [🧹 Cleanup & Safety](#-cleanup--safety)
- [⚠️ Disclaimer](#-disclaimer)
- [📦 Development & Contributing](#-development--contributing)
- [Building from Source](#building-from-source)
- [Contributing](#contributing)
- [Release Process](#release-process)
- [📚 Documentation](#-documentation)
- [🔧 Troubleshooting](#-troubleshooting)
- [🏆 Acknowledgments](#-acknowledgments)
- [📄 License](#-license)
- [⚠️ Legal Notice](#-legal-notice)
---
## ✨ Features
- **🎯 Donor Hardware Analysis**: Extract real PCIe device configurations and register maps
- **💾 Full 4 KB Config-Space Shadow**: Complete configuration space emulation with overlay RAM
- **🔄 MSI-X Table Replication**: Exact replication of MSI-X tables from donor devices
- **✂️ Capability Pruning**: Selective modification of capabilities that can't be faithfully emulated
- **🎲 Deterministic Variance Seeding**: Consistent hardware variance based on device serial number
- **📊 Behavioral Profiling**: Capture dynamic device behavior patterns for enhanced realism
- **🔧 Manufacturing Variance Simulation**: Add realistic timing jitter and parameter variations
- **⚡ Advanced SystemVerilog Generation**: Comprehensive PCIe device controller with modular architecture
- **🐳 Automated Build Pipeline**: Containerized synthesis and bit-stream generation
- **🔌 USB-JTAG Flashing**: Direct firmware deployment to DMA boards
- **🖥️ Interactive TUI**: Modern text-based interface with real-time monitoring and guided workflows
- **📦 Professional Packaging**: Easy installation via pip with proper dependency management
## 🚀 Quick Start
### Installation
```bash
# Basic installation
pip install pcileechfwgenerator
# With TUI support (recommended)
pip install pcileechfwgenerator[tui]
# Development installation
pip install pcileechfwgenerator[dev]
# Get sudo wrapper
wget https://raw.githubusercontent.com/ramseymcgrath/PCILeechFWGenerator/refs/heads/main/install-sudo-wrapper.sh
# Install sudo wrapper scripts (recommended for TUI and build commands)
./install-sudo-wrapper.sh
```
### Usage
```bash
# Interactive TUI interface (recommended)
pcileech-tui-sudo # Use sudo wrapper (preserves Python path)
# OR
sudo pcileech-tui # Direct sudo (may have module import issues)
# Command line interface
pcileech-generate
# Direct build command
pcileech-build-sudo --bdf 0000:03:00.0 --board 75t # Use sudo wrapper
# OR
sudo pcileech-build --bdf 0000:03:00.0 --board 75t # Direct sudo
```
### Device Suitability Indicators
In the TUI, devices are evaluated for firmware generation compatibility:
| Indicator | Meaning |
|-----------|---------|
| ✅ | **Suitable**: Device is compatible and not bound to a driver |
| ⚠️ | **Suitable with Warning**: Device is compatible but bound to a driver |
| ❌ | **Not Suitable**: Device is not compatible for firmware generation |
A device is considered suitable when its suitability score is ≥ 0.7 and it has no compatibility issues.
## 📋 Requirements
### Software
This is primarily tested in Linux, with some fiddling you could probably get it to work on Windows too.
| Tool | Why you need it | Install |
|------|----------------|---------|
| Vivado Studio | Synthesis & bitstream generation | Download from Xilinx (any 2022.2+ release) |
| Podman | Rootless container runtime for the build sandbox | See installation instructions below |
| Python ≥ 3.9 | Hostside orchestrator ([`generate.py`](generate.py)) | Distro package (python3) |
| λConcept usbloader | USB flashing utility for Screamerclass boards | See installation instructions below |
| pciutils, usbutils | lspci / lsusb helpers | Available in most Linux distributions |
> **⚠️ Security Notice**
> Never build firmware on the same operating system you plan to run the attack from. Use a separate Linux box.
### Hardware
| Component | Notes |
|-----------|-------|
| Donor PCIe card | Any inexpensive NIC, sound, or capture card works. One donor → one firmware. Destroy or quarantine the donor after extraction. |
| DMA board | Supported Artix7 DMA boards (35T, 75T, 100T). Must expose the Screamer USBJTAG port. |
## 🛠️ Installation & Setup
### Method 1: pip Installation (Recommended)
```bash
# Install with TUI support
pip install pcileechfwgenerator[tui]
# Basic installation (CLI only)
pip install pcileechfwgenerator
# Development installation
git clone https://github.com/ramseymcgrath/PCILeechFWGenerator
cd PCILeechFWGenerator
pip install -e .[dev]
```
### Method 2: Manual Installation
```bash
# Clone repository
git clone https://github.com/ramseymcgrath/PCILeechFWGenerator
cd PCILeechFWGenerator
# Install system dependencies
sudo ./install.sh
# Install Python dependencies
pip install -r requirements-tui.txt # For TUI support
# OR
pip install -r requirements.txt # Basic installation
```
Relogin or run `newgrp` afterwards so rootless Podman picks up subuid/subgid mappings.
## 🎮 Usage
### Interactive TUI Mode (Recommended)
The modern text-based interface provides guided workflows and real-time monitoring:
```bash
# Launch TUI (after pip installation)
sudo pcileech-tui
# Or from source
sudo python3 tui_generate.py
```
**TUI Features:**
- 🖥️ **Visual device browser** with enhanced PCIe device information
- ⚙️ **Guided configuration** with validation and profile management
- 📊 **Real-time build monitoring** with progress tracking and resource usage
- 🔍 **Intelligent error guidance** with suggested fixes
- 📡 **System status monitoring** for Podman, Vivado, USB devices, and more
See [`docs/TUI_README.md`](docs/TUI_README.md) for detailed TUI documentation.
### Command Line Mode
For automated workflows or scripting:
```bash
# Basic generation (interactive device selection)
sudo pcileech-generate
# Direct build with specific device (local build by default)
sudo pcileech-build --bdf 0000:03:00.0 --board 75t
# Build using donor_dump kernel module (opt-in)
sudo pcileech-build --bdf 0000:03:00.0 --board 75t --use-donor-dump
# Build using a previously saved donor information file
sudo pcileech-build --bdf 0000:03:00.0 --board 75t --donor-info-file /path/to/donor_info.json
# For manual donor dump generation, see the Manual Donor Dump Guide:
# docs/MANUAL_DONOR_DUMP.md
# Advanced generation with enhanced features
sudo pcileech-build --bdf 0000:03:00.0 --board 75t --advanced-sv
# Device-specific generation with behavior profiling
sudo pcileech-build --bdf 0000:03:00.0 --board 75t --advanced-sv \
--device-type network --enable-behavior-profiling
# Selective feature control
sudo pcileech-build --bdf 0000:03:00.0 --board 75t --advanced-sv \
--disable-power-management --disable-performance-counters
```
**Note:** When using container builds, the system will automatically build the required container image (`dma-fw`) if it doesn't exist. This happens during the first run and requires an internet connection to download base images.
**Output:** `output/firmware.bin` (FPGA bitstream ready for flashing).
### Legacy Mode (Source Installation)
```bash
# Traditional workflow (still supported)
sudo python3 generate.py
```
## 🔌 Flashing the DMA Board
> **Note:** These steps can run on the same machine or a different PC.
1. Power down, install the DMA card, and remove the donor.
2. Connect the USBJTAG port.
3. Flash:
```bash
usbloader -f output/firmware.bin # autodetects Screamer VID:PID 1d50:6130
```
If multiple λConcept boards are attached, add `--vidpid <vid:pid>`.
## 🚀 Advanced Features
### Full 4 KB Config-Space Shadow
The configuration space shadow BRAM implementation provides a complete 4 KB PCI Express configuration space in block RAM (BRAM) on the FPGA. This is a critical component for PCIe device emulation, as it allows the PCILeech firmware to accurately respond to configuration space accesses from the host system.
**Key Capabilities:**
- **Complete Configuration Space**: Full 4 KB configuration space shadow in BRAM
- **Dual-Port Access**: Simultaneous read/write operations for improved performance
- **Overlay RAM**: Dedicated storage for writable fields (Command/Status registers)
- **Donor Initialization**: Automatic initialization from donor device configuration data
- **PCIe Compatibility**: Little-endian format compatible with PCIe specification
**Integration Benefits:**
- **Enhanced Realism**: Complete configuration space emulation for better device mimicry
- **Improved Compatibility**: Support for extended capabilities and configuration registers
- **Flexible Access**: Proper handling of read-only and read-write fields
- **Seamless Integration**: Works with MSI-X table replication and capability pruning
**Usage:**
```bash
# Enabled by default in all builds
sudo pcileech-build --bdf 0000:03:00.0 --board 75t
```
For more details, see [CONFIG_SPACE_SHADOW.md](docs/CONFIG_SPACE_SHADOW.md).
### MSI-X Table Replication
The MSI-X table replication feature extends the PCILeech FPGA firmware generator to accurately replicate the MSI-X capability structure from donor devices. This enables the emulated device to support advanced interrupt handling capabilities, which is critical for high-performance device emulation.
**Key Capabilities:**
- **Automatic Parsing**: Extract MSI-X capability structure from donor configuration space
- **Parameterized Implementation**: SystemVerilog implementation of MSI-X table and PBA
- **Byte-Enable Writes**: Support for granular writes to MSI-X table entries
- **Interrupt Delivery**: Complete interrupt delivery logic with masking support
- **BAR Integration**: Seamless integration with the BAR controller for memory-mapped access
**Integration Benefits:**
- **Enhanced Compatibility**: Support for devices that rely on MSI-X interrupts
- **Improved Performance**: Efficient interrupt handling for high-performance devices
- **Realistic Behavior**: Accurate emulation of MSI-X interrupt delivery and masking
- **Flexible Configuration**: Support for different table sizes and configurations
**Usage:**
```bash
# Automatically enabled when MSI-X capability is detected in donor device
sudo pcileech-build --bdf 0000:03:00.0 --board 75t
```
For more details, see [MSIX_TABLE_REPLICATION.md](docs/MSIX_TABLE_REPLICATION.md).
### Capability Pruning
The PCI capability pruning feature extends the PCILeech FPGA firmware generator to analyze and selectively modify or remove PCI capabilities that cannot be faithfully emulated. This ensures that the emulated device presents a consistent and compatible configuration space to the host system.
**Key Capabilities:**
- **Automatic Analysis**: Identify and categorize all capabilities in the donor's configuration space
- **Selective Modification**: Prune or modify capabilities based on emulation feasibility
- **Chain Preservation**: Maintain capability chain integrity after modifications
- **Comprehensive Coverage**: Support for both standard and extended capabilities
**Pruning Rules:**
- **ASPM / L1SS**: Clear ASPM bits and remove L1 PM Substates capability
- **OBFF / LTR**: Zero OBFF & LTR bits and remove LTR capability
- **SR-IOV**: Remove SR-IOV capability entirely
- **Advanced PM**: Keep only D0/D3hot power states and clear PME support bits
**Integration Benefits:**
- **Improved Stability**: Prevent issues with capabilities that can't be properly emulated
- **Enhanced Compatibility**: Better compatibility with different host systems
- **Reduced Detection Risk**: Remove capabilities that might reveal the emulation
- **Focused Emulation**: Concentrate on accurately emulating supported capabilities
**Usage:**
```bash
# Enabled by default in all builds
sudo pcileech-build --bdf 0000:03:00.0 --board 75t
# Disable capability pruning if needed
sudo pcileech-build --bdf 0000:03:00.0 --board 75t --disable-capability-pruning
```
For more details, see [CAPABILITY_PRUNING.md](docs/CAPABILITY_PRUNING.md).
### Deterministic Variance Seeding
The deterministic variance seeding feature ensures that two builds of the same donor device at the same commit fall in the same timing band. This provides consistent behavior across builds while still maintaining realistic hardware variance.
**Key Capabilities:**
- **Deterministic Seed Generation**: Generate consistent seeds based on device serial number (DSN) and build revision
- **Consistent Variance**: Same donor device and build revision produce identical variance parameters
- **Device-Specific Variance**: Different donor devices produce different variance parameters
- **Build-Specific Variance**: Different build revisions produce different variance parameters
**Integration Benefits:**
- **Reproducible Builds**: Consistent behavior across builds of the same donor device
- **Enhanced Realism**: Realistic hardware variance that's unique to each donor device
- **Reduced Detection Risk**: Variance parameters that match the donor device's characteristics
- **Seamless Integration**: Works with manufacturing variance simulation and behavioral profiling
**Usage:**
```bash
# Automatically enabled when DSN is available in donor device
sudo pcileech-build --bdf 0000:03:00.0 --board 75t --advanced-sv
```
For more details, see [INTEGRATED_FEATURES.md](docs/INTEGRATED_FEATURES.md).
### Manufacturing Variance Simulation
The Manufacturing Variance Simulation feature adds realistic hardware variations to make generated firmware more authentic and harder to detect. This feature models real-world manufacturing tolerances and environmental conditions.
**Key Capabilities:**
- **Device Class Support**: Consumer, Enterprise, Industrial, and Automotive grade components with appropriate variance characteristics
- **Timing Variations**: Clock jitter, register access timing, and propagation delays
- **Environmental Modeling**: Temperature drift, power supply noise, and process variations
- **Integration**: Seamlessly integrates with behavior profiling for enhanced realism
**Device Classes:**
- `CONSUMER`: Standard consumer-grade tolerances (2-5% clock jitter)
- `ENTERPRISE`: Tighter enterprise-grade specifications (1-3% variations)
- `INDUSTRIAL`: Extended temperature and reliability ranges
- `AUTOMOTIVE`: Automotive-grade specifications with enhanced stability
**Usage:**
```bash
# Enable manufacturing variance (automatically applied with --advanced-sv)
python3 src/build.py --bdf 0000:03:00.0 --board 75t --advanced-sv
```
### Advanced SystemVerilog Generation
The Advanced SystemVerilog Generation feature provides a comprehensive, modular PCIe device controller with enterprise-grade capabilities and optimizations.
**Architecture Components:**
- **Modular Design**: Specialized components for power, error handling, and performance monitoring
- **Multiple Clock Domains**: Proper clock domain crossing with variance-aware timing
- **Device-Specific Logic**: Optimizations for Network, Storage, Graphics, and Audio devices
- **Comprehensive Integration**: All components work together seamlessly
**Power Management Features:**
- **PCIe Power States**: Full D0, D1, D2, D3hot, D3cold state support
- **ASPM (Active State Power Management)**: L0s, L1, L1.1, L1.2 link states
- **Dynamic Power Scaling**: Frequency and voltage scaling based on workload
- **Wake-on-LAN/Event**: Configurable wake event handling
**Error Handling & Recovery:**
- **Comprehensive Error Detection**: Correctable and uncorrectable error handling
- **Advanced Error Reporting (AER)**: Full PCIe AER implementation
- **Recovery Mechanisms**: Automatic error recovery and link retraining
- **Error Logging**: Detailed error tracking and reporting
**Performance Monitoring:**
- **Hardware Counters**: Transaction, bandwidth, and latency monitoring
- **Device-Specific Metrics**: Tailored counters for different device types
- **Real-time Monitoring**: Live performance data collection
- **Threshold-based Alerts**: Configurable performance thresholds
**Device-Specific Optimizations:**
- **Network Devices**: Multi-queue support, interrupt coalescing, checksum offload
- **Storage Devices**: Command queuing, wear leveling, power loss protection
- **Graphics Devices**: Memory bandwidth optimization, display timing
- **Audio Devices**: Low-latency processing, sample rate conversion
**Usage Examples:**
```bash
# Enable all advanced features
python3 src/build.py --bdf 0000:03:00.0 --board 75t --advanced-sv
# Network device with specific optimizations
python3 src/build.py --bdf 0000:03:00.0 --board 75t --advanced-sv --device-type network
# Disable specific features for minimal implementation
python3 src/build.py --bdf 0000:03:00.0 --board 75t --advanced-sv \
--disable-power-management --disable-error-handling
# Storage device with performance monitoring
python3 src/build.py --bdf 0000:03:00.0 --board 75t --advanced-sv \
--device-type storage --enable-behavior-profiling
```
### Behavioral Profiling
Dynamic behavior profiling captures real device operation patterns to enhance firmware realism by monitoring and analyzing how the donor device behaves during normal operation.
**Capabilities:**
- **Real-time Monitoring**: Captures live device register access patterns and timing
- **Pattern Analysis**: Identifies behavioral signatures, timing characteristics, and state transitions
- **Manufacturing Variance Integration**: Combines with variance simulation for enhanced realism
- **SystemVerilog Enhancement**: Automatically integrates behavioral data into generated code
- **Configurable Duration**: Adjustable profiling periods (default: 30 seconds)
**Key Benefits:**
- **Enhanced Realism**: Generated firmware mimics actual device behavior patterns
- **Improved Timing Accuracy**: Precise register access timing based on real-world measurements
- **Optimized Performance**: Device-specific optimizations based on observed behavior
- **Reduced Detection Risk**: More authentic behavioral signatures
**Usage:**
```bash
# Enable behavior profiling with custom duration
python3 src/build.py --bdf 0000:03:00.0 --board 75t \
--enable-behavior-profiling --profile-duration 30.0
# Enable profiling with device-specific optimizations
python3 src/build.py --bdf 0000:03:00.0 --board 75t --advanced-sv \
--device-type network --enable-behavior-profiling
```
### Command-Line Options
**Core Options:**
- `--bdf`: PCIe Bus:Device.Function identifier (required)
- `--board`: Target board type (35t, 75t, 100t) (required)
**Build Options:**
- `--container-engine`: Specify container engine to use (docker or podman, default: podman)
**Donor Device Options:**
- `--use-donor-dump`: Use the donor_dump kernel module (opt-in, not default)
- `--donor-info-file`: Path to a JSON file containing donor information from a previous run
**Advanced Features:**
- `--advanced-sv`: Enable advanced SystemVerilog generation
- `--device-type`: Specify device type (generic, network, storage, graphics, audio)
- `--enable-behavior-profiling`: Enable dynamic behavior profiling
- `--profile-duration`: Profiling duration in seconds (default: 30.0)
**Feature Control:**
- `--disable-power-management`: Disable power management features
- `--disable-error-handling`: Disable error handling features
- `--disable-performance-counters`: Disable performance monitoring
- `--disable-capability-pruning`: Disable capability pruning
**Analysis & Debugging:**
- `--save-analysis`: Save detailed analysis to specified file
- `--verbose`: Enable verbose output
- `--enhanced-timing`: Enable enhanced timing models (default: enabled)
## 🧹 Cleanup & Safety
- Rebind the donor back to its original driver if you keep it around.
- Keep the generated firmware private; it contains identifiers from the donor.
- Advanced features require appropriate privileges for hardware access.
## ⚠️ Disclaimer
This tool is intended for educational research and legitimate PCIe development purposes only. Users are responsible for ensuring compliance with all applicable laws and regulations. The authors assume no liability for misuse of this software.
## 📦 Development & Contributing
For development setup instructions, please see [`docs/DEVELOPMENT.md`](docs/DEVELOPMENT.md).
### Building from Source
```bash
# Build distributions
python -m build
# Install locally
pip install dist/*.whl
# Test installation
pcileech-generate --help
```
### Contributing
We welcome contributions! Please see [`CONTRIBUTING.md`](CONTRIBUTING.md) for detailed guidelines.
**Quick Start:**
1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Make your changes
4. Add tests for new functionality
5. Run the test suite (`pytest`)
6. Commit your changes (`git commit -m 'feat: add amazing feature'`)
7. Push to the branch (`git push origin feature/amazing-feature`)
8. Open a Pull Request
### Release Process
For maintainers releasing new versions:
```bash
# Automated release (recommended)
python scripts/release.py minor --release-notes "Add new TUI features and improvements"
# Manual release
python -m build
twine upload dist/*
```
## 📚 Documentation
- **[Integrated Features](docs/INTEGRATED_FEATURES.md)**: Comprehensive documentation of integrated features
- **[TUI Documentation](docs/TUI_README.md)**: Detailed TUI interface guide
- **[TUI Design Document](docs/TUI_Design_Document.md)**: Technical architecture
- **[Manual Donor Dump Guide](docs/MANUAL_DONOR_DUMP.md)**: Step-by-step guide for manually generating donor dumps
- **[Contributing Guide](CONTRIBUTING.md)**: Development and contribution guidelines
- **[Changelog](CHANGELOG.md)**: Version history and release notes
- **[API Reference](https://pcileechfwgenerator.readthedocs.io/)**: Complete API documentation
## 🔧 Troubleshooting
### Common Issues
**Installation Problems:**
```bash
# If pip installation fails
pip install --upgrade pip setuptools wheel
pip install pcileechfwgenerator[tui]
# For development installation issues
pip install -e .[dev]
```
**TUI Not Starting:**
```bash
# Check TUI dependencies
python -c "import textual; print('TUI dependencies OK')"
# Install TUI dependencies manually
pip install textual rich psutil watchdog
```
**Permission Issues:**
```bash
# Ensure proper permissions for PCIe operations
sudo usermod -a -G vfio $USER
sudo usermod -a -G dialout $USER # For USB-JTAG access
```
**Container Issues:**
```bash
# Check Podman installation
podman --version
# Verify rootless setup
podman info | grep rootless
# Test container build and dependencies
./scripts/build_container.sh --test
# Manual container dependency check
podman run --rm pcileechfwgenerator:latest python3 -c "import psutil, pydantic; print('Dependencies OK')"
# Check container file structure
podman run --rm pcileechfwgenerator:latest ls -la /app/src/
# Test with specific capabilities (recommended)
podman run --rm --cap-add=SYS_RAWIO --cap-add=SYS_ADMIN pcileechfwgenerator:latest echo "Capability test passed"
```
**Donor Dump Issues:**
```bash
# If donor_dump module fails to build or load
# See the Manual Donor Dump Guide for step-by-step instructions:
# docs/MANUAL_DONOR_DUMP.md
# Manual donor dump extraction (Linux)
sudo insmod src/donor_dump/donor_dump.ko bdf=0000:03:00.0
cat /proc/donor_dump > donor_info.txt
# Windows donor dump extraction
# Run PowerShell as Administrator
.\scripts\windows_donor_dump.ps1 -BDF "0000:03:00.0" -OutputFile "donor_info.json"
```
**Container Security Best Practices:**
- Use specific capabilities (`--cap-add=SYS_RAWIO --cap-add=SYS_ADMIN`) instead of `--privileged` when possible
- Always mount output directories to preserve generated files: `-v ./output:/app/output`
- The container runs as non-root user `appuser` by default for security
- Use the build script for automated testing: `./scripts/build_container.sh --test`
### Getting Help
- **GitHub Issues**: [Report bugs or request features](https://github.com/ramseymcgrath/PCILeechFWGenerator/issues)
- **GitHub Discussions**: [Community support and questions](https://github.com/ramseymcgrath/PCILeechFWGenerator/discussions)
- **Documentation**: Check the docs/ directory for detailed guides
## 🏆 Acknowledgments
- **λConcept**: For the excellent usbloader utility and Screamer hardware
- **Xilinx/AMD**: For Vivado synthesis tools
- **Textual**: For the modern TUI framework
- **PCILeech Community**: For feedback and contributions
## 📄 License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## ⚠️ Legal Notice
This tool is intended for educational research and legitimate PCIe development purposes only. Users are responsible for ensuring compliance with all applicable laws and regulations. The authors assume no liability for misuse of this software.
**Security Considerations:**
- Never build firmware on systems used for production or sensitive operations
- Use isolated build environments (Seperate dedicated hardware)
- Keep generated firmware private and secure
- Follow responsible disclosure practices for any security research
---
**Version 0.3.0** - Major release with integrated features for comprehensive PCIe device emulation
For educational research and legitimate PCIe development only. Misuse may violate laws and void warranties. The authors assume no liability.

13
SECURITY.md Normal file
View File

@ -0,0 +1,13 @@
# Security Policy
## Supported Versions
Use the latest version. This tool is as secure as it can be but please read and evaluate the source code yourself
## Reporting a Vulnerability
Use this section to tell people how to report a vulnerability.
Tell them where to go, how often they can expect to get an update on a
reported vulnerability, what to expect if the vulnerability is accepted or
declined, etc.

56
entrypoint.sh Normal file
View File

@ -0,0 +1,56 @@
#!/bin/bash
set -e
# Display usage information
if [ "$1" = "--help" ] || [ "$1" = "-h" ]; then
echo "PCILeech DMA Firmware Generator Container v2.0"
echo "Usage: podman run --rm -it --cap-add=SYS_RAWIO --cap-add=SYS_ADMIN \\"
echo " --device=/dev/vfio/GROUP --device=/dev/vfio/vfio \\"
echo " -v ./output:/app/output dma-fw \\"
echo " sudo python3 /app/src/build.py [OPTIONS]"
echo ""
echo "Required arguments:"
echo " --bdf XXXX:XX:XX.X PCI Bus:Device.Function (e.g., 0000:03:00.0)"
echo " --board XXt Target board (35t, 75t, or 100t)"
echo ""
echo "Advanced SystemVerilog options:"
echo " --advanced-sv Enable advanced SystemVerilog generation"
echo " --device-type TYPE Device type optimization (network, storage, graphics)"
echo " --enable-variance Enable manufacturing variance simulation"
echo " --disable-power-management Disable power management features"
echo " --enable-debug Enable debug features in generated code"
echo " --custom-config FILE Use custom configuration file"
echo ""
echo "Basic examples:"
echo " # Standard firmware generation (local build is default)"
echo " sudo python3 /app/src/build.py --bdf 0000:03:00.0 --board 75t"
echo ""
echo "Advanced examples:"
echo " # Basic advanced generation"
echo " sudo python3 /app/src/build.py --bdf 0000:03:00.0 --board 75t --advanced-sv"
echo ""
echo " # Device-specific with variance"
echo " sudo python3 /app/src/build.py --bdf 0000:03:00.0 --board 75t --advanced-sv --device-type network --enable-variance"
echo ""
echo " # Custom configuration"
echo " sudo python3 /app/src/build.py --bdf 0000:03:00.0 --board 75t --advanced-sv --device-type storage --disable-power-management"
echo ""
echo "Kernel Module (donor_dump, optional):"
echo " To use donor_dump (add --use-donor-dump flag), the module must be built on the host system:"
echo " 1. Copy /app/src/donor_dump/ to host"
echo " 2. Install kernel headers: apt-get install linux-headers-\$(uname -r)"
echo " 3. Build module: cd donor_dump && make"
echo " 4. Load module: sudo insmod donor_dump.ko bdf=XXXX:XX:XX.X"
echo ""
echo "Features:"
echo " - Basic PCILeech DMA firmware generation"
echo " - Advanced SystemVerilog code generation"
echo " - Manufacturing variance simulation"
echo " - Device-specific optimizations"
echo " - Power management controls"
echo " - Performance profiling and optimization"
exit 0
fi
# Execute the command
exec "$@"

9
generate.log Normal file
View File

@ -0,0 +1,9 @@
2025-06-02 09:36:45,700 - INFO - Starting PCILeech firmware generation process
2025-06-02 09:36:45,700 - ERROR - This script requires root privileges. Run with sudo.
2025-06-02 09:36:45,700 - ERROR - Fatal error during firmware generation: This script requires root privileges. Run with sudo.
2025-06-04 17:29:02,781 - INFO - Starting PCILeech firmware generation process
2025-06-04 17:29:02,781 - ERROR - This script requires root privileges. Run with sudo.
2025-06-04 17:29:02,782 - ERROR - Fatal error during firmware generation: This script requires root privileges. Run with sudo.
2025-06-04 17:29:13,984 - INFO - Starting PCILeech firmware generation process
2025-06-04 17:29:14,844 - ERROR - Error checking container image: Command 'podman images dma-fw --format '{{.Repository}}'' returned non-zero exit status 125.
2025-06-04 17:29:14,844 - ERROR - Fatal error during firmware generation: Error checking container image: Command 'podman images dma-fw --format '{{.Repository}}'' returned non-zero exit status 125.

View File

@ -14,9 +14,11 @@ Requires root privileges (sudo) for driver rebinding and VFIO operations.
"""
import argparse
import datetime
import logging
import os
import pathlib
import platform
import re
import shutil
import subprocess
@ -25,6 +27,17 @@ import textwrap
import time
from typing import Dict, List, Optional, Tuple
# Import donor dump manager
try:
from src.donor_dump_manager import DonorDumpError, DonorDumpManager
except ImportError:
DonorDumpManager = None
DonorDumpError = Exception
# Git repository information
PCILEECH_FPGA_REPO = "https://github.com/ufrisk/pcileech-fpga.git"
REPO_CACHE_DIR = os.path.expanduser("~/.cache/pcileech-fw-generator/repos")
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s - %(levelname)s - %(message)s",
@ -51,8 +64,27 @@ def run_command(cmd: str, **kwargs) -> str:
return subprocess.check_output(cmd, shell=True, text=True, **kwargs).strip()
def is_linux() -> bool:
"""Check if running on Linux."""
import platform
return platform.system().lower() == "linux"
def check_linux_requirement(operation: str) -> None:
"""Check if operation requires Linux and raise error if not available."""
if not is_linux():
raise RuntimeError(
f"{operation} requires Linux. "
f"Current platform: {platform.system()}. "
f"Please run this on a Linux system with VFIO support."
)
def list_pci_devices() -> List[Dict[str, str]]:
"""List all PCIe devices with their details."""
check_linux_requirement("PCIe device enumeration")
pattern = re.compile(
r"(?P<bdf>[0-9a-fA-F:.]+) .*?\["
r"(?P<class>[0-9a-fA-F]{4})\]: .*?\["
@ -87,6 +119,8 @@ def choose_device(devices: List[Dict[str, str]]) -> Dict[str, str]:
def get_current_driver(bdf: str) -> Optional[str]:
"""Get the current driver bound to a PCIe device."""
check_linux_requirement("Driver detection")
if not validate_bdf_format(bdf):
raise ValueError(
f"Invalid BDF format: {bdf}. Expected format: DDDD:BB:DD.F (e.g., 0000:03:00.0)"
@ -100,6 +134,8 @@ def get_current_driver(bdf: str) -> Optional[str]:
def get_iommu_group(bdf: str) -> str:
"""Get the IOMMU group for a PCIe device."""
check_linux_requirement("IOMMU group detection")
if not validate_bdf_format(bdf):
raise ValueError(
f"Invalid BDF format: {bdf}. Expected format: DDDD:BB:DD.F (e.g., 0000:03:00.0)"
@ -123,7 +159,7 @@ def list_usb_devices() -> List[Tuple[str, str]]:
)
if match:
vid_pid = match.group(1)
description = match.group(2)
description = match.group(2).strip() # Strip whitespace from description
devices.append((vid_pid, description))
return devices
@ -187,6 +223,8 @@ def bind_to_vfio(
bdf: str, vendor: str, device: str, original_driver: Optional[str]
) -> None:
"""Bind PCIe device to vfio-pci driver"""
check_linux_requirement("VFIO device binding")
if not validate_bdf_format(bdf):
raise ValueError(
f"Invalid BDF format: {bdf}. Expected format: DDDD:BB:DD.F (e.g., 0000:03:00.0)"
@ -195,7 +233,14 @@ def bind_to_vfio(
logger.info(
f"Binding device {bdf} to vfio-pci driver (current driver: {original_driver or 'none'})"
)
print("[*] Binding device to vfio-pci driver...")
if original_driver == "vfio-pci":
print(
"[*] Device already bound to vfio-pci driver, skipping binding process..."
)
logger.info(f"Device {bdf} already bound to vfio-pci, skipping binding process")
return
else:
print("[*] Binding device to vfio-pci driver...")
try:
# Check if vfio-pci driver is available
@ -206,20 +251,71 @@ def bind_to_vfio(
logger.error(error_msg)
raise RuntimeError(error_msg)
# Add device ID to vfio-pci
logger.debug(f"Adding device ID {vendor}:{device} to vfio-pci")
run_command(f"echo {vendor} {device} > /sys/bus/pci/drivers/vfio-pci/new_id")
# Check if device ID is already registered with vfio-pci
device_id_registered = False
try:
# Try to read the IDs file to check if our device ID is already registered
ids_file = "/sys/bus/pci/drivers/vfio-pci/ids"
if os.path.exists(ids_file):
with open(ids_file, "r") as f:
registered_ids = f.read()
if f"{vendor} {device}" in registered_ids:
logger.info(
f"Device ID {vendor}:{device} already registered with vfio-pci"
)
device_id_registered = True
except Exception as e:
logger.debug(f"Error checking registered device IDs: {e}")
# Continue with normal flow if we can't check
# Register device ID with vfio-pci if not already registered
if not device_id_registered:
logger.debug(f"Registering device ID {vendor}:{device} with vfio-pci")
try:
run_command(
f"echo {vendor} {device} > /sys/bus/pci/drivers/vfio-pci/new_id"
)
logger.info(
f"Successfully registered device ID {vendor}:{device} with vfio-pci"
)
except subprocess.CalledProcessError as e:
# If the error is "File exists", the device ID is already registered, which is fine
if "File exists" in str(e):
logger.info(
f"Device ID {vendor}:{device} already registered with vfio-pci"
)
else:
# Re-raise if it's a different error
logger.error(f"Failed to register device ID: {e}")
raise
# Unbind from current driver if present
if original_driver:
logger.debug(f"Unbinding from current driver: {original_driver}")
run_command(f"echo {bdf} > /sys/bus/pci/devices/{bdf}/driver/unbind")
try:
run_command(f"echo {bdf} > /sys/bus/pci/devices/{bdf}/driver/unbind")
logger.info(f"Successfully unbound {bdf} from {original_driver}")
except subprocess.CalledProcessError as e:
logger.warning(f"Failed to unbind from current driver: {e}")
# Continue anyway, as the bind might still work
# Bind to vfio-pci
logger.debug(f"Binding {bdf} to vfio-pci")
run_command(f"echo {bdf} > /sys/bus/pci/drivers/vfio-pci/bind")
logger.info(f"Successfully bound {bdf} to vfio-pci")
try:
run_command(f"echo {bdf} > /sys/bus/pci/drivers/vfio-pci/bind")
logger.info(f"Successfully bound {bdf} to vfio-pci")
print("[✓] Device successfully bound to vfio-pci driver")
except subprocess.CalledProcessError as e:
# Check if device is already bound to vfio-pci despite the error
current_driver = get_current_driver(bdf)
if current_driver == "vfio-pci":
logger.info(
f"Device {bdf} is already bound to vfio-pci despite bind command error"
)
print("[✓] Device is already bound to vfio-pci driver")
else:
logger.error(f"Failed to bind to vfio-pci: {e}")
raise
except subprocess.CalledProcessError as e:
error_msg = f"Failed to bind device to vfio-pci: {e}"
@ -309,7 +405,7 @@ def run_build_container(
logger.error(error_msg)
raise RuntimeError(error_msg)
# Build the build.py command with all arguments
# Build the build.py command with all arguments - use modular build system if available
build_cmd_parts = [f"sudo python3 /app/build.py --bdf {bdf} --board {board}"]
# Add advanced features arguments
@ -372,6 +468,96 @@ def run_build_container(
raise RuntimeError(error_msg) from e
def ensure_git_repo(repo_url: str, local_dir: str, update: bool = False) -> str:
"""
Ensure that the git repository is available locally.
Args:
repo_url (str): URL of the git repository
local_dir (str): Local directory to clone/pull the repository
update (bool): Whether to update the repository if it already exists
Returns:
str: Path to the local repository
"""
# Create cache directory if it doesn't exist
os.makedirs(os.path.dirname(local_dir), exist_ok=True)
# Check if repository already exists as a valid git repository
if os.path.exists(os.path.join(local_dir, ".git")):
logger.info(f"Repository already exists at {local_dir}")
# Update repository if requested
if update:
try:
logger.info(f"Updating repository at {local_dir}")
print(f"[*] Updating repository at {local_dir}")
# Get current directory
current_dir = os.getcwd()
# Change to repository directory
os.chdir(local_dir)
# Pull latest changes
result = subprocess.run(
"git pull",
shell=True,
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
# Change back to original directory
os.chdir(current_dir)
logger.info(f"Repository updated successfully: {result.stdout.strip()}")
print(f"[✓] Repository updated successfully")
except subprocess.CalledProcessError as e:
logger.warning(f"Failed to update repository: {e.stderr}")
print(f"[!] Warning: Failed to update repository: {e.stderr}")
else:
# Check if directory exists but is not a git repository
if os.path.exists(local_dir):
logger.info(f"Directory exists but is not a git repository: {local_dir}")
print(f"[*] Removing existing directory: {local_dir}")
# Remove the directory to allow fresh clone
import shutil
try:
shutil.rmtree(local_dir)
except Exception as e:
logger.warning(f"Failed to remove directory: {e}")
print(f"[!] Warning: Failed to remove directory: {e}")
# Continue anyway, git clone might still work
# Clone repository
try:
logger.info(f"Cloning repository {repo_url} to {local_dir}")
print(f"[*] Cloning repository {repo_url} to {local_dir}")
result = subprocess.run(
f"git clone {repo_url} {local_dir}",
shell=True,
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
logger.info(f"Repository cloned successfully")
print(f"[✓] Repository cloned successfully")
except subprocess.CalledProcessError as e:
error_msg = f"Failed to clone repository: {e.stderr}"
logger.error(error_msg)
raise RuntimeError(error_msg)
# Return path to repository
return local_dir
def validate_environment() -> None:
"""Validate that the environment is properly set up."""
if os.geteuid() != 0:
@ -379,12 +565,67 @@ def validate_environment() -> None:
logger.error(error_msg)
raise RuntimeError(error_msg)
# Check if git is available
if shutil.which("git") is None:
error_msg = "Git not found in PATH. Please install Git first."
logger.error(error_msg)
raise RuntimeError(error_msg)
# Check if Podman is available
if shutil.which("podman") is None:
error_msg = "Podman not found in PATH. Please install Podman first."
logger.error(error_msg)
raise RuntimeError(error_msg)
# Check if Vivado is available
try:
# Import vivado_utils from src directory
from pathlib import Path
sys.path.insert(0, str(Path(__file__).parent / "src"))
from src.vivado_utils import find_vivado_installation
vivado_info = find_vivado_installation()
if vivado_info:
logger.info(
f"Found Vivado {vivado_info['version']} at {vivado_info['path']}"
)
print(f"[✓] Vivado {vivado_info['version']} detected")
else:
logger.warning("Vivado not found. It will be used from the container.")
print("[!] Warning: Vivado not found locally. Will use container version.")
except ImportError:
logger.warning("Could not import vivado_utils. Skipping Vivado check.")
print("[!] Warning: Skipping Vivado check.")
# Check if container image exists
try:
result = run_command("podman images dma-fw --format '{{.Repository}}'")
if "dma-fw" not in result:
# Container image not found, try to build it
logger.info("Container image 'dma-fw' not found. Building it now...")
print("[*] Container image 'dma-fw' not found. Building it now...")
try:
build_result = subprocess.run(
"podman build -t dma-fw .",
shell=True,
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
logger.info("Container image built successfully")
print("[✓] Container image built successfully")
except subprocess.CalledProcessError as e:
error_msg = f"Failed to build container image: {e.stderr}"
logger.error(error_msg)
raise RuntimeError(error_msg)
except Exception as e:
error_msg = f"Error checking container image: {str(e)}"
logger.error(error_msg)
raise RuntimeError(error_msg)
def main() -> int:
"""Main entry point for the firmware generator"""
@ -395,6 +636,11 @@ def main() -> int:
logger.info("Starting PCILeech firmware generation process")
validate_environment()
# Ensure pcileech-fpga repository is available
repo_dir = os.path.join(REPO_CACHE_DIR, "pcileech-fpga")
pcileech_fpga_dir = ensure_git_repo(PCILEECH_FPGA_REPO, repo_dir, update=False)
logger.info(f"Using pcileech-fpga repository at {pcileech_fpga_dir}")
# Parse command line arguments
parser = argparse.ArgumentParser(
description="Generate DMA firmware from donor PCIe device",
@ -436,7 +682,22 @@ def main() -> int:
parser.add_argument(
"--board",
choices=["35t", "75t", "100t"],
choices=[
# Original boards
"35t",
"75t",
"100t",
# CaptainDMA boards
"pcileech_75t484_x1",
"pcileech_35t484_x1",
"pcileech_35t325_x4",
"pcileech_35t325_x1",
"pcileech_100t484_x1",
# Other boards
"pcileech_enigma_x1",
"pcileech_squirrel",
"pcileech_pciescreamer_xc7a35",
],
default="35t",
help="Target FPGA board type (default: 35t/Squirrel)",
)
@ -489,6 +750,19 @@ def main() -> int:
help="Duration for behavior profiling in seconds (default: 30)",
)
# Donor dump functionality
parser.add_argument(
"--donor-dump",
action="store_true",
help="Extract donor device parameters using kernel module before generation",
)
parser.add_argument(
"--auto-install-headers",
action="store_true",
help="Automatically install kernel headers if missing (for donor dump)",
)
args = parser.parse_args()
# Check if TUI mode is requested
@ -556,6 +830,65 @@ def main() -> int:
print(f"IOMMU group: {iommu_group}")
print(f"Current driver: {original_driver or 'none'}")
# Extract donor device parameters if requested
donor_info = None
if args.donor_dump:
if DonorDumpManager is None:
error_msg = "Donor dump functionality not available. Check src/donor_dump_manager.py"
logger.error(error_msg)
print(f"[✗] {error_msg}")
return 1
try:
print(f"\n[•] Extracting donor device parameters for {bdf}...")
logger.info(f"Starting donor dump extraction for {bdf}")
dump_manager = DonorDumpManager()
donor_info = dump_manager.setup_module(
bdf, auto_install_headers=args.auto_install_headers
)
print("[✓] Donor device parameters extracted successfully")
logger.info("Donor dump extraction completed successfully")
# Log key parameters
key_params = [
"vendor_id",
"device_id",
"class_code",
"bar_size",
"mpc",
"mpr",
]
for param in key_params:
if param in donor_info:
logger.info(f" {param}: {donor_info[param]}")
# Save donor info to file for container use
import json
donor_info_path = pathlib.Path("output/donor_info.json")
donor_info_path.parent.mkdir(exist_ok=True)
with open(donor_info_path, "w") as f:
json.dump(donor_info, f, indent=2)
logger.info(f"Donor info saved to {donor_info_path}")
except DonorDumpError as e:
error_msg = f"Donor dump failed: {e}"
logger.error(error_msg)
print(f"[✗] {error_msg}")
# Ask user if they want to continue without donor dump
response = input("Continue without donor dump? [y/N]: ").strip().lower()
if response not in ["y", "yes"]:
return 1
print("[•] Continuing without donor dump...")
except Exception as e:
error_msg = f"Unexpected error during donor dump: {e}"
logger.error(error_msg)
print(f"[✗] {error_msg}")
return 1
# Bind device to vfio-pci
bind_to_vfio(bdf, vendor, device, original_driver)

45
install-sudo-wrapper.sh Normal file
View File

@ -0,0 +1,45 @@
#!/bin/bash
# Install the pcileech sudo wrapper scripts
# Determine the installation directory
INSTALL_DIR="/usr/local/bin"
if [ ! -w "$INSTALL_DIR" ]; then
# If we don't have write permission to /usr/local/bin, use ~/.local/bin
INSTALL_DIR="$HOME/.local/bin"
mkdir -p "$INSTALL_DIR"
fi
# Copy the wrapper scripts to the installation directory
cp pcileech-tui-sudo "$INSTALL_DIR/"
chmod +x "$INSTALL_DIR/pcileech-tui-sudo"
cp pcileech-build-sudo "$INSTALL_DIR/"
chmod +x "$INSTALL_DIR/pcileech-build-sudo"
echo "Installed pcileech sudo wrappers to $INSTALL_DIR"
echo "You can now run the TUI with sudo using: pcileech-tui-sudo"
echo "You can now run the build with sudo using: pcileech-build-sudo"
# Add the directory to PATH if it's not already there
if [[ ":$PATH:" != *":$INSTALL_DIR:"* ]]; then
echo "Adding $INSTALL_DIR to your PATH"
# Determine which shell configuration file to use
SHELL_CONFIG=""
if [ -f "$HOME/.bashrc" ]; then
SHELL_CONFIG="$HOME/.bashrc"
elif [ -f "$HOME/.zshrc" ]; then
SHELL_CONFIG="$HOME/.zshrc"
elif [ -f "$HOME/.profile" ]; then
SHELL_CONFIG="$HOME/.profile"
fi
if [ -n "$SHELL_CONFIG" ]; then
echo "export PATH=\"\$PATH:$INSTALL_DIR\"" >> "$SHELL_CONFIG"
echo "Added $INSTALL_DIR to your PATH in $SHELL_CONFIG"
echo "Please restart your terminal or run 'source $SHELL_CONFIG' to apply changes"
else
echo "Could not find a shell configuration file to update PATH"
echo "Please manually add $INSTALL_DIR to your PATH"
fi
fi

Binary file not shown.

18
pcileech-build-sudo Normal file
View File

@ -0,0 +1,18 @@
#!/bin/bash
# Wrapper script to run pcileech-build with sudo while preserving Python path
# Get the path to the installed pcileech-build script
PCILEECH_BUILD_PATH=$(which pcileech-build)
if [ -z "$PCILEECH_BUILD_PATH" ]; then
echo "Error: pcileech-build not found in PATH"
exit 1
fi
# Get Python site-packages directories
PYTHON_USER_SITE=$(python3 -m site --user-site)
PYTHON_SITE_PACKAGES=$(python3 -c "import site; print(':'.join(site.getsitepackages()))")
# Run with sudo, preserving the PYTHONPATH
echo "Running pcileech-build with sudo and preserved Python paths..."
sudo PYTHONPATH="$PYTHONPATH:$PYTHON_USER_SITE:$PYTHON_SITE_PACKAGES" "$PCILEECH_BUILD_PATH" "$@"

18
pcileech-tui-sudo Normal file
View File

@ -0,0 +1,18 @@
#!/bin/bash
# Wrapper script to run pcileech-tui with sudo while preserving Python path
# Get the path to the installed pcileech-tui script
PCILEECH_TUI_PATH=$(which pcileech-tui)
if [ -z "$PCILEECH_TUI_PATH" ]; then
echo "Error: pcileech-tui not found in PATH"
exit 1
fi
# Get Python site-packages directories
PYTHON_USER_SITE=$(python3 -m site --user-site)
PYTHON_SITE_PACKAGES=$(python3 -c "import site; print(':'.join(site.getsitepackages()))")
# Run with sudo, preserving the PYTHONPATH
echo "Running pcileech-tui with sudo and preserved Python paths..."
sudo PYTHONPATH="$PYTHONPATH:$PYTHON_USER_SITE:$PYTHON_SITE_PACKAGES" "$PCILEECH_TUI_PATH"

View File

@ -3,16 +3,16 @@ requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "pcileech-fw-generator"
name = "pcileechfwgenerator"
dynamic = ["version"]
description = "Generate spoofed PCIe DMA firmware from real donor hardware"
readme = "README.md"
license = "MIT"
authors = [
{name = "Ramsey McGrath", email = "ramseymcgrath@gmail.com"}
{name = "Ramsey McGrath", email = "ramsey@voltcyclone.info"}
]
maintainers = [
{name = "Ramsey McGrath", email = "ramseymcgrath@gmail.com"}
{name = "Ramsey McGrath", email = "ramsey@voltcyclone.info"}
]
keywords = [
"pcie", "dma", "firmware", "fpga", "hardware", "security",
@ -64,6 +64,7 @@ test = [
"pytest>=7.0.0",
"pytest-cov>=4.0.0",
"pytest-mock>=3.10.0",
"pytest-asyncio>=0.21.0",
]
[project.urls]
@ -76,7 +77,7 @@ Changelog = "https://github.com/ramseymcgrath/PCILeechFWGenerator/blob/main/CHAN
[project.scripts]
pcileech-generate = "src.generate_cli:main"
pcileech-tui = "src.tui_cli:main"
pcileech-build = "src.build:main"
pcileech-build = "src.build_cli:main"
[tool.setuptools]
packages = ["src", "src.tui", "src.tui.core", "src.tui.models", "src.scripts"]
@ -175,4 +176,4 @@ exclude_lines = [
"if __name__ == .__main__.:",
"class .*\\bProtocol\\):",
"@(abc\\.)?abstractmethod",
]
]

View File

@ -3,7 +3,7 @@ testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
addopts =
addopts =
--verbose
--tb=short
--strict-markers
@ -15,6 +15,8 @@ addopts =
--cov-report=xml:coverage.xml
--cov-fail-under=10
--durations=10
asyncio_mode = auto
asyncio_default_fixture_loop_scope = function
markers =
unit: Unit tests
integration: Integration tests

View File

@ -10,6 +10,7 @@ pytest>=7.0.0
pytest-cov>=4.0.0
pytest-mock>=3.10.0
pytest-xdist>=3.0.0 # Parallel test execution
pytest-asyncio>=0.21.0 # Asyncio support for tests
# Code formatting and linting
black>=23.0.0

View File

@ -5,6 +5,7 @@ pytest-mock>=3.10.0
pytest-xdist>=3.0.0
pytest-timeout>=2.1.0
pytest-benchmark>=4.0.0
pytest-asyncio>=0.21.0
# Code quality and linting
flake8>=6.0.0
@ -37,4 +38,7 @@ pyfakefs>=5.2.0
# Additional utilities
tox>=4.0.0
coverage[toml]>=7.0.0
coverage[toml]>=7.0.0
# HTTP client for GitHub integration
requests>=2.28.0

View File

@ -2,7 +2,7 @@
# Install with: pip install -r requirements-tui.txt
# Core TUI framework
textual>=0.45.0
textual==3.3.0
# Rich text rendering and formatting
rich>=13.0.0

View File

@ -5,4 +5,7 @@
psutil>=5.9.0
# Configuration validation and serialization
pydantic>=2.0.0
pydantic>=2.0.0
# Async file I/O for modular build system
aiofiles>=23.0.0

View File

@ -201,6 +201,39 @@ def run_performance_tests():
return False
def run_external_tests(verbose=False):
"""Run external example validation tests."""
print("\n" + "=" * 60)
print("EXTERNAL EXAMPLE TESTS")
print("=" * 60)
# Specify the external example test files directly
external_test_files = [
"tests/test_tcl_validation.py",
"tests/test_sv_validation.py",
"tests/test_external_integration.py",
"tests/test_build_integration.py",
]
cmd_parts = ["pytest"]
cmd_parts.extend(external_test_files)
if verbose:
cmd_parts.append("-v")
cmd_parts.extend(["--tb=short", "--junit-xml=junit-external.xml"])
cmd = " ".join(cmd_parts)
try:
run_command(cmd, "Running external example tests")
print("[✓] External example tests passed")
return True
except subprocess.CalledProcessError:
print("[✗] External example tests failed")
return False
def run_legacy_tests():
"""Run the original test_enhancements.py for backward compatibility."""
print("\n" + "=" * 60)
@ -279,6 +312,7 @@ def generate_test_report(results):
"htmlcov/index.html",
"junit-unit.xml",
"junit-integration.xml",
"junit-external.xml",
"benchmark.json",
"bandit-report.json",
]
@ -307,6 +341,7 @@ Examples:
python run_tests.py --ci # CI mode (no interactive tests)
python run_tests.py --coverage # With coverage reporting
python run_tests.py --performance # Performance tests only
python run_tests.py --external # External example tests only
""",
)
@ -327,6 +362,9 @@ Examples:
parser.add_argument(
"--legacy", action="store_true", help="Run legacy enhancement tests only"
)
parser.add_argument(
"--external", action="store_true", help="Run external example tests only"
)
parser.add_argument(
"--no-quality", action="store_true", help="Skip code quality checks"
)
@ -336,7 +374,15 @@ Examples:
# Set default to quick if no specific mode is selected
if not any(
[args.quick, args.full, args.ci, args.performance, args.security, args.legacy]
[
args.quick,
args.full,
args.ci,
args.performance,
args.security,
args.legacy,
args.external,
]
):
args.quick = True
@ -372,8 +418,12 @@ Examples:
if args.performance or args.full:
results["Performance Tests"] = run_performance_tests()
# External example tests
if args.external or args.full or args.ci:
results["External Example Tests"] = run_external_tests(verbose=args.verbose)
# Unit tests
if args.quick or args.full or args.ci or not args.performance:
if args.quick or args.full or args.ci or not (args.performance or args.external):
results["Unit Tests"] = run_unit_tests(
coverage=args.coverage or args.full or args.ci, verbose=args.verbose
)