上传文件至 /
4.0 Beta版本 代码库重构、msix 改进、更好的兼容性消息传递 添加 force，修复不匹配的 vars 修复 tcl 格式化 状态机代码和 tui 选项中的错误修复
This commit is contained in:
parent
50cfb36ed2
commit
ab8dc65733
|
|
@ -15,6 +15,7 @@ AUTHORS*
|
|||
# Build artifacts and output
|
||||
output/
|
||||
build/
|
||||
!src/build.py
|
||||
dist/
|
||||
*.bin
|
||||
*.bit
|
||||
|
|
|
|||
|
|
@ -0,0 +1,262 @@
|
|||
# 📝 Changelog
|
||||
|
||||
[](https://badge.fury.io/py/pcileech-fw-generator)
|
||||
[](https://pypi.org/project/pcileech-fw-generator/)
|
||||
[](https://opensource.org/licenses/MIT)
|
||||
|
||||
All notable changes to the PCILeech Firmware Generator will be documented in this file.
|
||||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v0.1.2.html).
|
||||
|
||||
---
|
||||
|
||||
## 📑 Table of Contents
|
||||
|
||||
- [Version 0.3.1 (2025-06-09)](#020---2025-06-09)
|
||||
- [Version 0.3.1 (2025-01-02)](#0110---2025-01-02)
|
||||
- [Release Notes](#release-notes)
|
||||
- [Backward Compatibility](#backward-compatibility)
|
||||
- [Future Roadmap](#future-roadmap)
|
||||
|
||||
---
|
||||
|
||||
## [0.3.1] - 2025-06-10
|
||||
|
||||
### ✨ Added
|
||||
- **🧩 Feature Integration**: Comprehensive integration of all major features
|
||||
- Integrated documentation in `docs/INTEGRATED_FEATURES.md`
|
||||
- Comprehensive integration tests in `tests/test_feature_integration.py`
|
||||
- Seamless interoperation between all components
|
||||
- **💾 Full 4 KB Config-Space Shadow in BRAM**: Complete configuration space emulation
|
||||
- Full 4 KB configuration space shadow in BRAM
|
||||
- Dual-port access for simultaneous read/write operations
|
||||
- Overlay RAM for writable fields (Command/Status registers)
|
||||
- Initialization from donor device configuration data
|
||||
- Little-endian format compatible with PCIe specification
|
||||
- **🔄 Auto-Replicate MSI-X Table**: Exact MSI-X table replication
|
||||
- Automatic parsing of MSI-X capability structure from donor configuration space
|
||||
- Parameterized SystemVerilog implementation of MSI-X table and PBA
|
||||
- Support for byte-enable granularity writes
|
||||
- Interrupt delivery logic with masking support
|
||||
- Integration with the existing BAR controller and configuration space shadow
|
||||
- **✂️ Capability Pruning**: Selective capability modification
|
||||
- Automatic analysis of standard and extended PCI capabilities
|
||||
- Categorization of capabilities based on emulation feasibility
|
||||
- Selective pruning of unsupported capabilities
|
||||
- Modification of partially supported capabilities
|
||||
- Preservation of capability chain integrity
|
||||
- **🎲 Deterministic Variance Seeding**: Consistent hardware variance
|
||||
- Deterministic seed generation based on device serial number (DSN) and build revision
|
||||
- Consistent variance parameters for the same donor device and build revision
|
||||
- Different variance parameters for different donor devices or build revisions
|
||||
- Support for different device classes with appropriate variance ranges
|
||||
- **🏗️ Build Process**: Enhanced to support all integrated features
|
||||
- Improved donor dump extraction with full 4 KB configuration space
|
||||
- Added capability pruning step to the build process
|
||||
- Added MSI-X table parameter extraction and integration
|
||||
- Added deterministic variance seeding based on DSN and build revision
|
||||
- **📋 Enhanced Logging**: Improved logging for all integrated features
|
||||
- Added detailed logging for capability pruning
|
||||
- Added MSI-X table parameter logging
|
||||
- Added variance parameter logging
|
||||
- Added integration status summary at the end of the build process
|
||||
- **📚 Documentation**: Comprehensive documentation for all integrated features
|
||||
- Added `docs/INTEGRATED_FEATURES.md` with detailed integration information
|
||||
- Updated feature-specific documentation with integration details
|
||||
- Added troubleshooting information for integrated features
|
||||
- **🔌 MSI-X Table Integration**: Fixed issues with MSI-X table integration
|
||||
- Corrected MSI-X table parameter extraction from configuration space
|
||||
- Fixed MSI-X table and PBA memory mapping in BAR controller
|
||||
- Improved error handling for MSI-X capability parsing
|
||||
- **🧩 Capability Chain Integrity**: Fixed issues with capability chain integrity
|
||||
- Ensured proper next pointer updates when removing capabilities
|
||||
- Fixed extended capability chain traversal and modification
|
||||
- Improved error handling for capability chain manipulation
|
||||
- **⏱️ Timing Consistency**: Fixed issues with timing consistency
|
||||
- Ensured deterministic variance seeding produces consistent results
|
||||
- Fixed timing parameter calculation and application
|
||||
- Improved error handling for variance parameter generation
|
||||
|
||||
## [0.3.1] - 2025-06-09
|
||||
|
||||
### ✨ Added
|
||||
- **💾 Option-ROM Passthrough**: Complete Option-ROM replication from donor devices
|
||||
- Extracts Option-ROM from donor PCI devices using Linux sysfs interface
|
||||
- Supports two implementation modes:
|
||||
- Mode A: BAR 5 Window (pure FPGA implementation)
|
||||
- Mode B: External SPI Flash (for larger ROMs)
|
||||
- Handles legacy 16-bit config cycles for ROM access
|
||||
- Includes caching for improved performance
|
||||
- Configurable ROM size and source
|
||||
- **🔧 Build System Integration**: Enhanced build process for Option-ROM support
|
||||
- Added command-line arguments for enabling and configuring Option-ROM feature
|
||||
- Automatic ROM extraction during build process
|
||||
- Support for using pre-extracted ROM files
|
||||
- Build-time selection between implementation modes
|
||||
- **🧪 Testing Infrastructure**: Comprehensive test suite for Option-ROM functionality
|
||||
- Unit tests for Option-ROM extraction and handling
|
||||
- Support for different ROM sizes and formats
|
||||
- Validation of ROM signature and content
|
||||
|
||||
### 🔄 Changed
|
||||
- **🔢 Version Bump**: Incremented to v0.3.1 to reflect significant Option-ROM feature addition
|
||||
- **🏗️ Build Process**: Updated to support Option-ROM integration
|
||||
- **📋 Enhanced Logging**: Improved logging for Option-ROM extraction and processing
|
||||
|
||||
## [Unreleased] - Build Process Improvements
|
||||
|
||||
### 🔄 Changed
|
||||
- **🏗️ Local Build Default**: Changed local build to be the default process
|
||||
- Local builds now run by default (no container required)
|
||||
- Container builds now require explicit opt-in with `--use-donor-dump`
|
||||
- Improved error handling for local build scenarios
|
||||
- Enhanced documentation for local build workflows
|
||||
- **🔧 Container Engine Options**: Added support for multiple container engines
|
||||
- Added new `--container-engine` option to specify engine preference
|
||||
- Podman is now the default container engine
|
||||
- Docker remains fully supported as an alternative option
|
||||
- Automatic detection of available container engines
|
||||
- **🔍 Vivado Location Validation**: Enhanced Vivado detection and validation
|
||||
- Improved cross-platform Vivado installation detection
|
||||
- Added support for environment variables (XILINX_VIVADO)
|
||||
- Automatic version detection and compatibility checking
|
||||
- Detailed error messages for missing or incompatible installations
|
||||
|
||||
### 🔧 Fixed
|
||||
- **🔌 VFIO Device Binding**: Fixed an issue where binding a device already bound to vfio-pci would fail
|
||||
- Added detection for devices already bound to vfio-pci
|
||||
- Improved error handling during the binding process
|
||||
- Added comprehensive test cases for this edge case
|
||||
- **📦 Container Dependency Installation**: Fixed missing Python dependencies in container build
|
||||
- Added proper `pip install` commands for `requirements.txt` and `requirements-tui.txt`
|
||||
- Fixed import errors for `psutil`, `pydantic`, and other required packages
|
||||
- **📁 Container File Structure**: Corrected file paths and directory structure
|
||||
- Fixed `build.py` path from `/app/build.py` to `/app/src/build.py`
|
||||
- Updated all container usage examples and documentation
|
||||
- **🔒 Container Security Improvements**: Enhanced security posture
|
||||
- Replaced `--privileged` with specific capabilities (`--cap-add=SYS_RAWIO --cap-add=SYS_ADMIN`)
|
||||
- Maintained non-root user execution while preserving functionality
|
||||
- **✅ Container Health Checks**: Improved dependency validation
|
||||
- Enhanced health check to validate Python imports
|
||||
- Added comprehensive dependency testing
|
||||
|
||||
### ✨ Added
|
||||
- **🔨 Container Build Script**: New automated build and test script
|
||||
- Added `scripts/build_container.sh` with comprehensive testing
|
||||
- Supports both Podman and Docker container engines
|
||||
- Includes security validation and usage examples
|
||||
- **🚀 Container CI Pipeline**: Automated container testing workflow
|
||||
- Added `.github/workflows/container-ci.yml` for continuous integration
|
||||
- Tests container build, dependencies, security, and integration
|
||||
- Validates file structure and user permissions
|
||||
|
||||
### 📚 Improved
|
||||
- **📖 Documentation Updates**: Enhanced container usage documentation
|
||||
- Updated `podman_demo.md` with security best practices
|
||||
- Added troubleshooting section for container issues
|
||||
- Included capability-based security examples
|
||||
|
||||
### 🗂️ Changed
|
||||
- **📦 Container File Inclusion**: Updated `.dockerignore` configuration
|
||||
- Removed exclusion of `src/tui/` components
|
||||
- Included necessary requirements files
|
||||
- Optimized build context for better performance
|
||||
|
||||
---
|
||||
|
||||
### 🚀 Installation
|
||||
```bash
|
||||
# Basic installation
|
||||
pip install pcileech-fw-generator
|
||||
|
||||
# With TUI support
|
||||
pip install pcileech-fw-generator[tui]
|
||||
|
||||
# Development installation
|
||||
pip install pcileech-fw-generator[dev]
|
||||
```
|
||||
|
||||
### 🎮 Usage
|
||||
```bash
|
||||
# Command line interface (traditional)
|
||||
pcileech-generate
|
||||
|
||||
# Interactive TUI interface (new)
|
||||
pcileech-tui
|
||||
|
||||
# Direct build command
|
||||
pcileech-build --bdf 0000:03:00.0 --board 75t
|
||||
```
|
||||
|
||||
## [1.0.0] - 2024-12-01
|
||||
|
||||
### ✨ Added
|
||||
- Initial release of PCILeech Firmware Generator
|
||||
- Basic command-line interface for firmware generation
|
||||
- Donor hardware analysis and configuration extraction
|
||||
- Containerized build pipeline with Vivado integration
|
||||
- USB-JTAG flashing support for DMA boards
|
||||
- Basic SystemVerilog generation for PCIe devices
|
||||
- Podman-based isolated build environment
|
||||
|
||||
### 🎯 Features
|
||||
- PCIe device enumeration and selection
|
||||
- Configuration space extraction from donor hardware
|
||||
- FPGA bitstream generation for Artix-7 boards
|
||||
- Automated driver binding and VFIO operations
|
||||
- Basic logging and error handling
|
||||
|
||||
---
|
||||
|
||||
## 📋 Release Notes
|
||||
|
||||
### 🚀 v0.3.1 Highlights
|
||||
|
||||
This release integrates all major features of the PCILeech FPGA firmware generator, providing a comprehensive solution for PCIe device emulation. The integration ensures that all features work together seamlessly, providing a more realistic and functional emulation experience.
|
||||
|
||||
Key improvements include:
|
||||
- **💾 Full 4 KB Config-Space Shadow**: Complete configuration space emulation with overlay RAM for writable fields
|
||||
- **🔄 MSI-X Table Replication**: Exact replication of MSI-X tables from donor devices
|
||||
- **✂️ Capability Pruning**: Selective modification of capabilities that can't be faithfully emulated
|
||||
- **🎲 Deterministic Variance Seeding**: Consistent hardware variance based on device serial number and build revision
|
||||
|
||||
### 🚀 v0.3.1 Highlights
|
||||
|
||||
This release introduces the Option-ROM passthrough feature, allowing the PCILeech FPGA firmware to faithfully replicate the Option-ROM of donor PCI devices. This enables advanced functionality such as UEFI boot support and device-specific initialization.
|
||||
|
||||
Key improvements include:
|
||||
- **💾 Complete Option-ROM Replication**: Extract and replicate Option-ROMs from donor devices
|
||||
- **🔀 Dual Implementation Modes**: Choose between pure FPGA (BAR window) or SPI flash implementations
|
||||
- **🔌 Legacy ROM Support**: Proper handling of legacy 16-bit config cycles for ROM access
|
||||
- **🛠️ Flexible Configuration**: Command-line options for ROM source, size, and implementation mode
|
||||
|
||||
### 🚀 v0.3.1 Highlights
|
||||
|
||||
This major release introduces a modern, interactive TUI that transforms the user experience while maintaining full backward compatibility with the original command-line interface. The TUI provides guided workflows, real-time monitoring, and intelligent error handling that makes firmware generation more accessible and reliable.
|
||||
|
||||
Key improvements include:
|
||||
- **🎯 Zero Learning Curve**: Intuitive interface guides users through the entire process
|
||||
- **📊 Real-time Feedback**: Live monitoring of build progress and system resources
|
||||
- **🛡️ Error Prevention**: Validation and checks prevent common configuration mistakes
|
||||
- **📦 Professional Packaging**: Easy installation via pip with proper dependency management
|
||||
|
||||
### 🔄 Backward Compatibility
|
||||
|
||||
All existing command-line workflows continue to work unchanged. The new integrated features are designed to be backward compatible with existing workflows, ensuring a smooth transition for users.
|
||||
|
||||
### 🔮 Future Roadmap
|
||||
|
||||
- Web-based interface for remote build management
|
||||
- Enhanced device compatibility and detection
|
||||
- Advanced firmware customization options
|
||||
- Integration with additional FPGA toolchains
|
||||
- Cloud-based build services
|
||||
|
||||
## ⚠️ Disclaimer
|
||||
|
||||
This tool is intended for educational research and legitimate PCIe development purposes only. Users are responsible for ensuring compliance with all applicable laws and regulations. The authors assume no liability for misuse of this software.
|
||||
|
||||
---
|
||||
|
||||
**Version 0.3.1** - Major release with integrated features for comprehensive PCIe device emulation
|
||||
|
|
@ -0,0 +1,408 @@
|
|||
# 🤝 Contributing to PCILeech Firmware Generator
|
||||
|
||||
[](https://badge.fury.io/py/pcileech-fw-generator)
|
||||
[](https://pypi.org/project/pcileech-fw-generator/)
|
||||
[](https://opensource.org/licenses/MIT)
|
||||
[](https://github.com/ramseymcgrath/PCILeechFWGenerator/actions)
|
||||
|
||||
Thank you for your interest in contributing to the PCILeech Firmware Generator! This document provides guidelines and information for contributors.
|
||||
|
||||
---
|
||||
|
||||
## 📑 Table of Contents
|
||||
|
||||
- [📜 Code of Conduct](#-code-of-conduct)
|
||||
- [🚀 Getting Started](#-getting-started)
|
||||
- [Prerequisites](#prerequisites)
|
||||
- [Development Setup](#development-setup)
|
||||
- [📋 Contributing Guidelines](#-contributing-guidelines)
|
||||
- [Types of Contributions](#types-of-contributions)
|
||||
- [Bug Reports](#bug-reports)
|
||||
- [Feature Requests](#feature-requests)
|
||||
- [🔄 Development Workflow](#-development-workflow)
|
||||
- [Branch Strategy](#branch-strategy)
|
||||
- [Making Changes](#making-changes)
|
||||
- [Commit Message Format](#commit-message-format)
|
||||
- [🧪 Testing](#-testing)
|
||||
- [Test Structure](#test-structure)
|
||||
- [Writing Tests](#writing-tests)
|
||||
- [Running Tests](#running-tests)
|
||||
- [💻 Code Style](#-code-style)
|
||||
- [Python Style Guide](#python-style-guide)
|
||||
- [Formatting Tools](#formatting-tools)
|
||||
- [Pre-commit Hooks](#pre-commit-hooks)
|
||||
- [📚 Documentation](#-documentation)
|
||||
- [Code Documentation](#code-documentation)
|
||||
- [Documentation Style](#documentation-style)
|
||||
- [📤 Submitting Changes](#-submitting-changes)
|
||||
- [Pull Request Process](#pull-request-process)
|
||||
- [Pull Request Template](#pull-request-template)
|
||||
- [Review Process](#review-process)
|
||||
- [📦 Release Process](#-release-process)
|
||||
- [Version Management](#version-management)
|
||||
- [Release Steps](#release-steps)
|
||||
- [Distribution](#distribution)
|
||||
- [❓ Getting Help](#-getting-help)
|
||||
- [Communication Channels](#communication-channels)
|
||||
- [Development Resources](#development-resources)
|
||||
- [🏆 Recognition](#-recognition)
|
||||
- [⚠️ Disclaimer](#️-disclaimer)
|
||||
|
||||
---
|
||||
|
||||
## 📜 Code of Conduct
|
||||
|
||||
This project adheres to a code of conduct that promotes a welcoming and inclusive environment. By participating, you are expected to uphold this code.
|
||||
|
||||
## 🚀 Getting Started
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Python 3.9 or higher
|
||||
- Git
|
||||
- Podman or Docker
|
||||
- Vivado (for FPGA synthesis)
|
||||
- Linux environment (required for PCIe operations)
|
||||
|
||||
### Development Setup
|
||||
|
||||
1. **Fork and Clone**
|
||||
```bash
|
||||
git clone https://github.com/yourusername/PCILeechFWGenerator.git
|
||||
cd PCILeechFWGenerator
|
||||
```
|
||||
|
||||
2. **Create Virtual Environment**
|
||||
```bash
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate # On Windows: venv\Scripts\activate
|
||||
```
|
||||
|
||||
3. **Install Development Dependencies**
|
||||
```bash
|
||||
pip install -r requirements-dev.txt
|
||||
```
|
||||
|
||||
4. **Install Pre-commit Hooks**
|
||||
```bash
|
||||
pre-commit install
|
||||
```
|
||||
|
||||
5. **Verify Installation**
|
||||
```bash
|
||||
python -m pytest tests/
|
||||
```
|
||||
|
||||
## 📋 Contributing Guidelines
|
||||
|
||||
### Types of Contributions
|
||||
|
||||
We welcome several types of contributions:
|
||||
|
||||
- **Bug Reports**: Help us identify and fix issues
|
||||
- **Feature Requests**: Suggest new functionality
|
||||
- **Code Contributions**: Implement features or fix bugs
|
||||
- **Documentation**: Improve or add documentation
|
||||
- **Testing**: Add or improve test coverage
|
||||
|
||||
### Bug Reports
|
||||
|
||||
When reporting bugs, please include:
|
||||
|
||||
- **Environment Details**: OS, Python version, hardware setup
|
||||
- **Steps to Reproduce**: Clear, step-by-step instructions
|
||||
- **Expected vs Actual Behavior**: What should happen vs what does happen
|
||||
- **Error Messages**: Full error output and logs
|
||||
- **Hardware Configuration**: PCIe devices, DMA boards, etc.
|
||||
|
||||
### Feature Requests
|
||||
|
||||
For feature requests, please provide:
|
||||
|
||||
- **Use Case**: Why is this feature needed?
|
||||
- **Proposed Solution**: How should it work?
|
||||
- **Alternatives Considered**: Other approaches you've thought about
|
||||
- **Implementation Ideas**: Technical approach if you have one
|
||||
|
||||
## 🔄 Development Workflow
|
||||
|
||||
### Branch Strategy
|
||||
|
||||
- `main`: Stable release branch
|
||||
- `develop`: Integration branch for new features
|
||||
- `feature/feature-name`: Feature development branches
|
||||
- `bugfix/issue-description`: Bug fix branches
|
||||
- `hotfix/critical-fix`: Critical fixes for production
|
||||
|
||||
### Making Changes
|
||||
|
||||
1. **Create Feature Branch**
|
||||
```bash
|
||||
git checkout -b feature/your-feature-name
|
||||
```
|
||||
|
||||
2. **Make Changes**
|
||||
- Write code following our style guidelines
|
||||
- Add tests for new functionality
|
||||
- Update documentation as needed
|
||||
|
||||
3. **Test Changes**
|
||||
```bash
|
||||
# Run full test suite
|
||||
python -m pytest tests/
|
||||
|
||||
# Run specific tests
|
||||
python -m pytest tests/test_specific_module.py
|
||||
|
||||
# Run with coverage
|
||||
python -m pytest --cov=src tests/
|
||||
```
|
||||
|
||||
4. **Commit Changes**
|
||||
```bash
|
||||
git add .
|
||||
git commit -m "feat: add new TUI feature for device selection"
|
||||
```
|
||||
|
||||
### Commit Message Format
|
||||
|
||||
We use conventional commits for clear history:
|
||||
|
||||
```
|
||||
<type>(<scope>): <description>
|
||||
|
||||
[optional body]
|
||||
|
||||
[optional footer]
|
||||
```
|
||||
|
||||
**Types:**
|
||||
- `feat`: New feature
|
||||
- `fix`: Bug fix
|
||||
- `docs`: Documentation changes
|
||||
- `style`: Code style changes (formatting, etc.)
|
||||
- `refactor`: Code refactoring
|
||||
- `test`: Adding or updating tests
|
||||
- `chore`: Maintenance tasks
|
||||
|
||||
**Examples:**
|
||||
```
|
||||
feat(tui): add real-time build progress monitoring
|
||||
fix(build): resolve SystemVerilog generation error for network devices
|
||||
docs(readme): update installation instructions for TUI
|
||||
test(core): add unit tests for device manager
|
||||
```
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
### Test Structure
|
||||
|
||||
```
|
||||
tests/
|
||||
├── unit/ # Unit tests for individual modules
|
||||
├── integration/ # Integration tests for workflows
|
||||
├── fixtures/ # Test data and fixtures
|
||||
└── conftest.py # Pytest configuration
|
||||
```
|
||||
|
||||
### Writing Tests
|
||||
|
||||
- **Unit Tests**: Test individual functions and classes
|
||||
- **Integration Tests**: Test complete workflows
|
||||
- **Mock External Dependencies**: Use pytest-mock for external services
|
||||
- **Test Edge Cases**: Include error conditions and boundary cases
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# All tests
|
||||
python -m pytest
|
||||
|
||||
# Specific test file
|
||||
python -m pytest tests/test_build.py
|
||||
|
||||
# With coverage
|
||||
python -m pytest --cov=src --cov-report=html
|
||||
|
||||
# Parallel execution
|
||||
python -m pytest -n auto
|
||||
|
||||
# Specific markers
|
||||
python -m pytest -m "not slow"
|
||||
```
|
||||
|
||||
## 💻 Code Style
|
||||
|
||||
### Python Style Guide
|
||||
|
||||
We follow PEP 8 with some modifications:
|
||||
|
||||
- **Line Length**: 88 characters (Black default)
|
||||
- **Import Sorting**: Use isort with Black profile
|
||||
- **Type Hints**: Required for all public functions
|
||||
- **Docstrings**: Google style for all modules, classes, and functions
|
||||
|
||||
### Formatting Tools
|
||||
|
||||
```bash
|
||||
# Format code
|
||||
black src/ tests/
|
||||
|
||||
# Sort imports
|
||||
isort src/ tests/
|
||||
|
||||
# Lint code
|
||||
flake8 src/ tests/
|
||||
|
||||
# Type checking
|
||||
mypy src/
|
||||
```
|
||||
|
||||
### Pre-commit Hooks
|
||||
|
||||
Our pre-commit configuration automatically runs:
|
||||
- Black (code formatting)
|
||||
- isort (import sorting)
|
||||
- flake8 (linting)
|
||||
- mypy (type checking)
|
||||
- pytest (basic tests)
|
||||
|
||||
## 📚 Documentation
|
||||
|
||||
### Code Documentation
|
||||
|
||||
- **Docstrings**: All public functions, classes, and modules
|
||||
- **Type Hints**: All function parameters and return values
|
||||
- **Comments**: Explain complex logic and business rules
|
||||
- **README Updates**: Keep installation and usage instructions current
|
||||
|
||||
### Documentation Style
|
||||
|
||||
```python
|
||||
def generate_firmware(device_bdf: str, board_type: str) -> Path:
|
||||
"""Generate firmware for specified PCIe device.
|
||||
|
||||
Args:
|
||||
device_bdf: PCIe device Bus:Device.Function identifier
|
||||
board_type: Target FPGA board type (35t, 75t, 100t)
|
||||
|
||||
Returns:
|
||||
Path to generated firmware binary
|
||||
|
||||
Raises:
|
||||
DeviceNotFoundError: If specified device doesn't exist
|
||||
BuildError: If firmware generation fails
|
||||
|
||||
Example:
|
||||
>>> firmware_path = generate_firmware("0000:03:00.0", "75t")
|
||||
>>> print(f"Firmware generated: {firmware_path}")
|
||||
"""
|
||||
```
|
||||
|
||||
## 📤 Submitting Changes
|
||||
|
||||
### Pull Request Process
|
||||
|
||||
1. **Update Documentation**: Ensure README, docstrings, and comments are current
|
||||
2. **Add Tests**: Include tests for new functionality
|
||||
3. **Run Full Test Suite**: Ensure all tests pass
|
||||
4. **Update Changelog**: Add entry to CHANGELOG.md
|
||||
5. **Create Pull Request**: Use our PR template
|
||||
|
||||
### Pull Request Template
|
||||
|
||||
```markdown
|
||||
## Description
|
||||
Brief description of changes
|
||||
|
||||
## Type of Change
|
||||
- [ ] Bug fix
|
||||
- [ ] New feature
|
||||
- [ ] Breaking change
|
||||
- [ ] Documentation update
|
||||
|
||||
## Testing
|
||||
- [ ] Unit tests added/updated
|
||||
- [ ] Integration tests added/updated
|
||||
- [ ] Manual testing completed
|
||||
|
||||
## Checklist
|
||||
- [ ] Code follows style guidelines
|
||||
- [ ] Self-review completed
|
||||
- [ ] Documentation updated
|
||||
- [ ] Tests pass locally
|
||||
```
|
||||
|
||||
### Review Process
|
||||
|
||||
1. **Automated Checks**: CI/CD pipeline runs tests and linting
|
||||
2. **Code Review**: Maintainers review code and provide feedback
|
||||
3. **Testing**: Changes are tested in development environment
|
||||
4. **Approval**: At least one maintainer approval required
|
||||
5. **Merge**: Changes merged to appropriate branch
|
||||
|
||||
## 📦 Release Process
|
||||
|
||||
### Version Management
|
||||
|
||||
We use semantic versioning (MAJOR.MINOR.PATCH):
|
||||
|
||||
- **MAJOR**: Breaking changes
|
||||
- **MINOR**: New features (backward compatible)
|
||||
- **PATCH**: Bug fixes (backward compatible)
|
||||
|
||||
### Release Steps
|
||||
|
||||
1. **Update Version**: Increment version in `src/__version__.py`
|
||||
2. **Update Changelog**: Add release notes to CHANGELOG.md
|
||||
3. **Create Release Branch**: `release/vX.Y.Z`
|
||||
4. **Final Testing**: Comprehensive testing of release candidate
|
||||
5. **Tag Release**: Create git tag with version
|
||||
6. **Build Distribution**: Create wheel and source distributions
|
||||
7. **Publish**: Upload to PyPI
|
||||
8. **GitHub Release**: Create GitHub release with notes
|
||||
|
||||
### Distribution
|
||||
|
||||
```bash
|
||||
# Build distributions
|
||||
python -m build
|
||||
|
||||
# Check distributions
|
||||
twine check dist/*
|
||||
|
||||
# Upload to PyPI
|
||||
twine upload dist/*
|
||||
```
|
||||
|
||||
## ❓ Getting Help
|
||||
|
||||
### Communication Channels
|
||||
|
||||
- **GitHub Issues**: Bug reports and feature requests
|
||||
- **GitHub Discussions**: General questions and community discussion
|
||||
- **Email**: Direct contact for security issues
|
||||
|
||||
### Development Resources
|
||||
|
||||
- **Architecture Documentation**: See `docs/` directory
|
||||
- **API Reference**: Generated from docstrings
|
||||
- **Examples**: See `examples/` directory
|
||||
- **Test Cases**: See `tests/` directory
|
||||
|
||||
## 🏆 Recognition
|
||||
|
||||
Contributors are recognized in:
|
||||
- **CHANGELOG.md**: Release notes mention contributors
|
||||
- **GitHub Contributors**: Automatic recognition
|
||||
- **Release Notes**: Major contributions highlighted
|
||||
|
||||
## ⚠️ Disclaimer
|
||||
|
||||
This tool is intended for educational research and legitimate PCIe development purposes only. Users are responsible for ensuring compliance with all applicable laws and regulations. The authors assume no liability for misuse of this software.
|
||||
|
||||
---
|
||||
|
||||
Thank you for contributing to PCILeech Firmware Generator!
|
||||
|
||||
**Version 0.3.1** - Major release with TUI interface and professional packaging
|
||||
14
Makefile
14
Makefile
|
|
@ -1,6 +1,6 @@
|
|||
# Makefile for PCILeech Firmware Generator
|
||||
|
||||
.PHONY: help clean install install-dev test lint format build build-pypi upload-test upload-pypi release
|
||||
.PHONY: help clean install install-dev test lint format build build-pypi upload-test upload-pypi release container docker-build build-container
|
||||
|
||||
# Default target
|
||||
help:
|
||||
|
|
@ -24,6 +24,10 @@ help:
|
|||
@echo " upload-pypi - Upload to PyPI"
|
||||
@echo " release - Full release process"
|
||||
@echo ""
|
||||
@echo "Container:"
|
||||
@echo " container - Build container image (dma-fw)"
|
||||
@echo " docker-build - Build container image (default tag)"
|
||||
@echo ""
|
||||
@echo "Utilities:"
|
||||
@echo " check-deps - Check system dependencies"
|
||||
@echo " security - Run security scans"
|
||||
|
|
@ -86,10 +90,16 @@ security:
|
|||
bandit -r src/
|
||||
safety check
|
||||
|
||||
# Docker targets
|
||||
# Container targets
|
||||
container:
|
||||
./scripts/build_container.sh --tag dma-fw
|
||||
|
||||
docker-build:
|
||||
./scripts/build_container.sh
|
||||
|
||||
# Alias for container
|
||||
build-container: container
|
||||
|
||||
# Test package build
|
||||
test-build:
|
||||
@echo "Testing PyPI package build..."
|
||||
|
|
|
|||
|
|
@ -0,0 +1,13 @@
|
|||
# Security Policy
|
||||
|
||||
## Supported Versions
|
||||
|
||||
Use the latest version. This tool is as secure as it can be but please read and evaluate the source code yourself
|
||||
|
||||
## Reporting a Vulnerability
|
||||
|
||||
Use this section to tell people how to report a vulnerability.
|
||||
|
||||
Tell them where to go, how often they can expect to get an update on a
|
||||
reported vulnerability, what to expect if the vulnerability is accepted or
|
||||
declined, etc.
|
||||
BIN
generate.log
BIN
generate.log
Binary file not shown.
696
generate.py
696
generate.py
|
|
@ -6,7 +6,7 @@ This script:
|
|||
• Enumerates PCIe devices
|
||||
• Allows user to select a donor device
|
||||
• Re-binds donor to vfio-pci driver
|
||||
• Launches Podman container (image: dma-fw) that runs build.py
|
||||
• Launches Podman container (image: pcileech-fw-generator) that runs build.py
|
||||
• Optionally flashes output/firmware.bin with usbloader
|
||||
• Restores original driver afterwards
|
||||
|
||||
|
|
@ -38,6 +38,35 @@ except ImportError:
|
|||
PCILEECH_FPGA_REPO = "https://github.com/ufrisk/pcileech-fpga.git"
|
||||
REPO_CACHE_DIR = os.path.expanduser("~/.cache/pcileech-fw-generator/repos")
|
||||
|
||||
|
||||
def clear_python_cache():
|
||||
"""Clear Python bytecode cache files to ensure updated code is used."""
|
||||
import glob
|
||||
|
||||
cache_patterns = [
|
||||
"__pycache__",
|
||||
"src/__pycache__",
|
||||
"tests/__pycache__",
|
||||
"tests/tui/__pycache__",
|
||||
"src/tui/__pycache__",
|
||||
"src/tui/core/__pycache__",
|
||||
"src/tui/models/__pycache__",
|
||||
"src/scripts/__pycache__",
|
||||
]
|
||||
|
||||
for pattern in cache_patterns:
|
||||
for cache_dir in glob.glob(pattern):
|
||||
if os.path.exists(cache_dir):
|
||||
try:
|
||||
shutil.rmtree(cache_dir)
|
||||
print(f"[*] Cleared Python cache: {cache_dir}")
|
||||
except Exception as e:
|
||||
print(f"[!] Warning: Could not clear cache {cache_dir}: {e}")
|
||||
|
||||
|
||||
# Clear Python cache at startup to ensure updated code is used
|
||||
clear_python_cache()
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s - %(levelname)s - %(message)s",
|
||||
|
|
@ -59,9 +88,19 @@ def validate_bdf_format(bdf: str) -> bool:
|
|||
return bool(bdf_pattern.match(bdf))
|
||||
|
||||
|
||||
def run_command(cmd: str, **kwargs) -> str:
|
||||
"""Execute a shell command and return stripped output."""
|
||||
return subprocess.check_output(cmd, shell=True, text=True, **kwargs).strip()
|
||||
def run_command(cmd: str, timeout: int = 30, **kwargs) -> str:
|
||||
"""Execute a shell command and return stripped output with timeout and better error handling."""
|
||||
try:
|
||||
return subprocess.check_output(
|
||||
cmd, shell=True, text=True, timeout=timeout, **kwargs
|
||||
).strip()
|
||||
except subprocess.TimeoutExpired as e:
|
||||
logger.error(f"Command timed out after {timeout}s: {cmd}")
|
||||
raise RuntimeError(f"Command timed out: {cmd}") from e
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.error(f"Command failed (exit code {e.returncode}): {cmd}")
|
||||
logger.error(f"Command stderr: {e.stderr}")
|
||||
raise
|
||||
|
||||
|
||||
def is_linux() -> bool:
|
||||
|
|
@ -219,10 +258,89 @@ def flash_firmware(bitfile: pathlib.Path) -> None:
|
|||
raise RuntimeError(error_msg) from e
|
||||
|
||||
|
||||
def _validate_vfio_prerequisites() -> None:
|
||||
"""Validate VFIO prerequisites and system configuration."""
|
||||
# Check if VFIO modules are loaded
|
||||
vfio_modules = ["/sys/module/vfio", "/sys/module/vfio_pci"]
|
||||
loaded_modules = [mod for mod in vfio_modules if os.path.exists(mod)]
|
||||
|
||||
if not loaded_modules:
|
||||
error_msg = (
|
||||
"VFIO modules not loaded. Please load VFIO modules:\n"
|
||||
" sudo modprobe vfio\n"
|
||||
" sudo modprobe vfio-pci"
|
||||
)
|
||||
logger.error(error_msg)
|
||||
raise RuntimeError(error_msg)
|
||||
|
||||
# Check if vfio-pci driver is available
|
||||
if not os.path.exists("/sys/bus/pci/drivers/vfio-pci"):
|
||||
error_msg = (
|
||||
"vfio-pci driver not available. Ensure VFIO is enabled in kernel.\n"
|
||||
"Check: cat /boot/config-$(uname -r) | grep -i vfio"
|
||||
)
|
||||
logger.error(error_msg)
|
||||
raise RuntimeError(error_msg)
|
||||
|
||||
# Check if IOMMU is enabled
|
||||
try:
|
||||
dmesg_output = run_command("dmesg | grep -i iommu", timeout=10)
|
||||
if (
|
||||
"IOMMU enabled" not in dmesg_output
|
||||
and "AMD-Vi" not in dmesg_output
|
||||
and "Intel-IOMMU" not in dmesg_output
|
||||
):
|
||||
logger.warning(
|
||||
"IOMMU may not be enabled. Check BIOS settings and kernel parameters."
|
||||
)
|
||||
except Exception:
|
||||
logger.debug("Could not check IOMMU status from dmesg")
|
||||
|
||||
|
||||
def _check_device_in_use(bdf: str) -> bool:
|
||||
"""Check if device is currently in use by checking for open file descriptors."""
|
||||
try:
|
||||
# Check if device has any open file descriptors
|
||||
lsof_output = run_command(
|
||||
f"lsof /dev/vfio/* 2>/dev/null | grep -v COMMAND || true", timeout=5
|
||||
)
|
||||
if bdf in lsof_output:
|
||||
logger.warning(f"Device {bdf} may be in use by another process")
|
||||
return True
|
||||
except Exception:
|
||||
logger.debug("Could not check device usage with lsof")
|
||||
return False
|
||||
|
||||
|
||||
def _wait_for_device_state(
|
||||
bdf: str, expected_driver: Optional[str], max_retries: int = 5
|
||||
) -> bool:
|
||||
"""Wait for device to reach expected driver state with retries."""
|
||||
import time
|
||||
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
current_driver = get_current_driver(bdf)
|
||||
if current_driver == expected_driver:
|
||||
return True
|
||||
|
||||
if attempt < max_retries - 1:
|
||||
logger.debug(
|
||||
f"Device {bdf} not in expected state (current: {current_driver}, expected: {expected_driver}), retrying in 1s..."
|
||||
)
|
||||
time.sleep(1)
|
||||
except Exception as e:
|
||||
logger.debug(f"Error checking device state (attempt {attempt + 1}): {e}")
|
||||
if attempt < max_retries - 1:
|
||||
time.sleep(1)
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def bind_to_vfio(
|
||||
bdf: str, vendor: str, device: str, original_driver: Optional[str]
|
||||
) -> None:
|
||||
"""Bind PCIe device to vfio-pci driver"""
|
||||
"""Bind PCIe device to vfio-pci driver with enhanced error handling and validation."""
|
||||
check_linux_requirement("VFIO device binding")
|
||||
|
||||
if not validate_bdf_format(bdf):
|
||||
|
|
@ -230,31 +348,46 @@ def bind_to_vfio(
|
|||
f"Invalid BDF format: {bdf}. Expected format: DDDD:BB:DD.F (e.g., 0000:03:00.0)"
|
||||
)
|
||||
|
||||
# Validate vendor and device IDs
|
||||
if not re.match(r"^[0-9a-fA-F]{4}$", vendor):
|
||||
raise ValueError(f"Invalid vendor ID format: {vendor}. Expected 4-digit hex.")
|
||||
if not re.match(r"^[0-9a-fA-F]{4}$", device):
|
||||
raise ValueError(f"Invalid device ID format: {device}. Expected 4-digit hex.")
|
||||
|
||||
logger.info(
|
||||
f"Binding device {bdf} to vfio-pci driver (current driver: {original_driver or 'none'})"
|
||||
f"Binding device {bdf} (vendor:{vendor} device:{device}) to vfio-pci driver (current driver: {original_driver or 'none'})"
|
||||
)
|
||||
|
||||
# Early exit if already bound to vfio-pci
|
||||
if original_driver == "vfio-pci":
|
||||
print(
|
||||
"[*] Device already bound to vfio-pci driver, skipping binding process..."
|
||||
)
|
||||
logger.info(f"Device {bdf} already bound to vfio-pci, skipping binding process")
|
||||
return
|
||||
else:
|
||||
|
||||
print("[*] Binding device to vfio-pci driver...")
|
||||
|
||||
try:
|
||||
# Check if vfio-pci driver is available
|
||||
if not os.path.exists("/sys/bus/pci/drivers/vfio-pci"):
|
||||
error_msg = (
|
||||
"vfio-pci driver not available. Ensure VFIO is enabled in kernel."
|
||||
)
|
||||
# Validate VFIO prerequisites
|
||||
_validate_vfio_prerequisites()
|
||||
|
||||
# Check if device exists
|
||||
device_path = f"/sys/bus/pci/devices/{bdf}"
|
||||
if not os.path.exists(device_path):
|
||||
error_msg = f"PCIe device {bdf} not found in sysfs"
|
||||
logger.error(error_msg)
|
||||
raise RuntimeError(error_msg)
|
||||
|
||||
# Check if device is in use
|
||||
if _check_device_in_use(bdf):
|
||||
logger.warning(
|
||||
f"Device {bdf} appears to be in use, proceeding with caution"
|
||||
)
|
||||
|
||||
# Check if device ID is already registered with vfio-pci
|
||||
device_id_registered = False
|
||||
try:
|
||||
# Try to read the IDs file to check if our device ID is already registered
|
||||
ids_file = "/sys/bus/pci/drivers/vfio-pci/ids"
|
||||
if os.path.exists(ids_file):
|
||||
with open(ids_file, "r") as f:
|
||||
|
|
@ -264,60 +397,173 @@ def bind_to_vfio(
|
|||
f"Device ID {vendor}:{device} already registered with vfio-pci"
|
||||
)
|
||||
device_id_registered = True
|
||||
except Exception as e:
|
||||
except (OSError, IOError) as e:
|
||||
logger.debug(f"Error checking registered device IDs: {e}")
|
||||
# Continue with normal flow if we can't check
|
||||
|
||||
# Register device ID with vfio-pci if not already registered
|
||||
if not device_id_registered:
|
||||
logger.debug(f"Registering device ID {vendor}:{device} with vfio-pci")
|
||||
max_retries = 3
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
run_command(
|
||||
f"echo {vendor} {device} > /sys/bus/pci/drivers/vfio-pci/new_id"
|
||||
)
|
||||
# Use direct file writing instead of shell redirection to avoid I/O errors
|
||||
new_id_path = "/sys/bus/pci/drivers/vfio-pci/new_id"
|
||||
with open(new_id_path, "w") as f:
|
||||
f.write(f"{vendor} {device}\n")
|
||||
logger.info(
|
||||
f"Successfully registered device ID {vendor}:{device} with vfio-pci"
|
||||
)
|
||||
except subprocess.CalledProcessError as e:
|
||||
# If the error is "File exists", the device ID is already registered, which is fine
|
||||
if "File exists" in str(e):
|
||||
break
|
||||
except (OSError, IOError) as e:
|
||||
if (
|
||||
"File exists" in str(e)
|
||||
or "Invalid argument" in str(e)
|
||||
or "Device or resource busy" in str(e)
|
||||
):
|
||||
logger.info(
|
||||
f"Device ID {vendor}:{device} already registered with vfio-pci"
|
||||
f"Device ID {vendor}:{device} already registered with vfio-pci or busy"
|
||||
)
|
||||
break
|
||||
elif attempt < max_retries - 1:
|
||||
logger.warning(
|
||||
f"Failed to register device ID (attempt {attempt + 1}): {e}, retrying..."
|
||||
)
|
||||
import time
|
||||
|
||||
time.sleep(1)
|
||||
else:
|
||||
# Re-raise if it's a different error
|
||||
logger.error(f"Failed to register device ID: {e}")
|
||||
raise
|
||||
logger.error(
|
||||
f"Failed to register device ID after {max_retries} attempts: {e}"
|
||||
)
|
||||
raise RuntimeError(f"Failed to register device ID: {e}")
|
||||
|
||||
# Unbind from current driver if present
|
||||
if original_driver:
|
||||
logger.debug(f"Unbinding from current driver: {original_driver}")
|
||||
max_retries = 3
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
run_command(f"echo {bdf} > /sys/bus/pci/devices/{bdf}/driver/unbind")
|
||||
# Use direct file writing instead of shell redirection
|
||||
unbind_path = f"/sys/bus/pci/devices/{bdf}/driver/unbind"
|
||||
with open(unbind_path, "w") as f:
|
||||
f.write(f"{bdf}\n")
|
||||
logger.info(f"Successfully unbound {bdf} from {original_driver}")
|
||||
except subprocess.CalledProcessError as e:
|
||||
logger.warning(f"Failed to unbind from current driver: {e}")
|
||||
|
||||
# Wait for unbind to complete
|
||||
if _wait_for_device_state(bdf, None, max_retries=3):
|
||||
break
|
||||
elif attempt < max_retries - 1:
|
||||
logger.warning(
|
||||
f"Device still bound after unbind (attempt {attempt + 1}), retrying..."
|
||||
)
|
||||
import time
|
||||
|
||||
time.sleep(1)
|
||||
else:
|
||||
logger.warning(
|
||||
"Device may still be bound to original driver, continuing..."
|
||||
)
|
||||
break
|
||||
|
||||
except (OSError, IOError) as e:
|
||||
if "No such device" in str(e) or "No such file or directory" in str(
|
||||
e
|
||||
):
|
||||
logger.info(f"Device {bdf} already unbound")
|
||||
break
|
||||
elif attempt < max_retries - 1:
|
||||
logger.warning(
|
||||
f"Failed to unbind from current driver (attempt {attempt + 1}): {e}, retrying..."
|
||||
)
|
||||
import time
|
||||
|
||||
time.sleep(1)
|
||||
else:
|
||||
logger.warning(
|
||||
f"Failed to unbind from current driver after {max_retries} attempts: {e}"
|
||||
)
|
||||
# Continue anyway, as the bind might still work
|
||||
|
||||
# Bind to vfio-pci
|
||||
# Bind to vfio-pci with retries
|
||||
logger.debug(f"Binding {bdf} to vfio-pci")
|
||||
max_retries = 3
|
||||
bind_successful = False
|
||||
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
run_command(f"echo {bdf} > /sys/bus/pci/drivers/vfio-pci/bind")
|
||||
# Use direct file writing instead of shell redirection
|
||||
bind_path = "/sys/bus/pci/drivers/vfio-pci/bind"
|
||||
with open(bind_path, "w") as f:
|
||||
f.write(f"{bdf}\n")
|
||||
|
||||
# Verify binding was successful
|
||||
if _wait_for_device_state(bdf, "vfio-pci", max_retries=3):
|
||||
logger.info(f"Successfully bound {bdf} to vfio-pci")
|
||||
print("[✓] Device successfully bound to vfio-pci driver")
|
||||
except subprocess.CalledProcessError as e:
|
||||
# Check if device is already bound to vfio-pci despite the error
|
||||
bind_successful = True
|
||||
break
|
||||
elif attempt < max_retries - 1:
|
||||
logger.warning(
|
||||
f"Bind command succeeded but device not bound to vfio-pci (attempt {attempt + 1}), retrying..."
|
||||
)
|
||||
import time
|
||||
|
||||
time.sleep(1)
|
||||
|
||||
except (OSError, IOError) as e:
|
||||
if "Device or resource busy" in str(e):
|
||||
logger.warning(f"Device {bdf} is busy (attempt {attempt + 1})")
|
||||
if attempt < max_retries - 1:
|
||||
import time
|
||||
|
||||
time.sleep(2) # Longer wait for busy devices
|
||||
continue
|
||||
elif "No such device" in str(e) or "No such file or directory" in str(
|
||||
e
|
||||
):
|
||||
logger.error(f"Device {bdf} disappeared during binding")
|
||||
raise RuntimeError(f"Device {bdf} not found during binding")
|
||||
elif attempt < max_retries - 1:
|
||||
logger.warning(
|
||||
f"Failed to bind to vfio-pci (attempt {attempt + 1}): {e}, retrying..."
|
||||
)
|
||||
import time
|
||||
|
||||
time.sleep(1)
|
||||
else:
|
||||
# Final attempt failed, check if device is actually bound
|
||||
current_driver = get_current_driver(bdf)
|
||||
if current_driver == "vfio-pci":
|
||||
logger.info(
|
||||
f"Device {bdf} is already bound to vfio-pci despite bind command error"
|
||||
f"Device {bdf} is bound to vfio-pci despite bind command error"
|
||||
)
|
||||
print("[✓] Device is already bound to vfio-pci driver")
|
||||
print("[✓] Device is bound to vfio-pci driver")
|
||||
bind_successful = True
|
||||
break
|
||||
else:
|
||||
logger.error(f"Failed to bind to vfio-pci: {e}")
|
||||
raise
|
||||
logger.error(
|
||||
f"Failed to bind to vfio-pci after {max_retries} attempts: {e}"
|
||||
)
|
||||
raise RuntimeError(f"Failed to bind to vfio-pci: {e}")
|
||||
|
||||
except subprocess.CalledProcessError as e:
|
||||
if not bind_successful:
|
||||
error_msg = (
|
||||
f"Failed to bind device {bdf} to vfio-pci after {max_retries} attempts"
|
||||
)
|
||||
logger.error(error_msg)
|
||||
raise RuntimeError(error_msg)
|
||||
|
||||
# Final verification
|
||||
final_driver = get_current_driver(bdf)
|
||||
if final_driver != "vfio-pci":
|
||||
error_msg = (
|
||||
f"Device {bdf} not bound to vfio-pci (current driver: {final_driver})"
|
||||
)
|
||||
logger.error(error_msg)
|
||||
raise RuntimeError(error_msg)
|
||||
|
||||
except (OSError, IOError) as e:
|
||||
error_msg = f"Failed to bind device to vfio-pci: {e}"
|
||||
logger.error(error_msg)
|
||||
raise RuntimeError(error_msg) from e
|
||||
|
|
@ -328,7 +574,7 @@ def bind_to_vfio(
|
|||
|
||||
|
||||
def restore_original_driver(bdf: str, original_driver: Optional[str]) -> None:
|
||||
"""Restore the original driver binding for the PCIe device"""
|
||||
"""Restore the original driver binding for the PCIe device with enhanced error handling."""
|
||||
if not validate_bdf_format(bdf):
|
||||
logger.warning(f"Invalid BDF format during restore: {bdf}")
|
||||
return
|
||||
|
|
@ -339,11 +585,69 @@ def restore_original_driver(bdf: str, original_driver: Optional[str]) -> None:
|
|||
print("[*] Restoring original driver binding...")
|
||||
|
||||
try:
|
||||
# Check if device is still bound to vfio-pci before attempting unbind
|
||||
# Check if device exists
|
||||
device_path = f"/sys/bus/pci/devices/{bdf}"
|
||||
if not os.path.exists(device_path):
|
||||
logger.warning(
|
||||
f"Device {bdf} not found during restore, may have been removed"
|
||||
)
|
||||
return
|
||||
|
||||
# Check current driver state
|
||||
current_driver = get_current_driver(bdf)
|
||||
logger.debug(f"Current driver for {bdf}: {current_driver or 'none'}")
|
||||
|
||||
# Unbind from vfio-pci if currently bound
|
||||
if current_driver == "vfio-pci":
|
||||
logger.debug(f"Unbinding {bdf} from vfio-pci")
|
||||
run_command(f"echo {bdf} > /sys/bus/pci/drivers/vfio-pci/unbind")
|
||||
max_retries = 3
|
||||
unbind_successful = False
|
||||
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
# Use direct file writing instead of shell redirection
|
||||
unbind_path = "/sys/bus/pci/drivers/vfio-pci/unbind"
|
||||
with open(unbind_path, "w") as f:
|
||||
f.write(f"{bdf}\n")
|
||||
|
||||
# Wait for unbind to complete
|
||||
if _wait_for_device_state(bdf, None, max_retries=3):
|
||||
logger.info(f"Successfully unbound {bdf} from vfio-pci")
|
||||
unbind_successful = True
|
||||
break
|
||||
elif attempt < max_retries - 1:
|
||||
logger.warning(
|
||||
f"Device still bound to vfio-pci (attempt {attempt + 1}), retrying..."
|
||||
)
|
||||
import time
|
||||
|
||||
time.sleep(1)
|
||||
|
||||
except (OSError, IOError) as e:
|
||||
if "No such device" in str(e) or "No such file or directory" in str(
|
||||
e
|
||||
):
|
||||
logger.info(f"Device {bdf} already unbound from vfio-pci")
|
||||
unbind_successful = True
|
||||
break
|
||||
elif attempt < max_retries - 1:
|
||||
logger.warning(
|
||||
f"Failed to unbind from vfio-pci (attempt {attempt + 1}): {e}, retrying..."
|
||||
)
|
||||
import time
|
||||
|
||||
time.sleep(1)
|
||||
else:
|
||||
logger.warning(
|
||||
f"Failed to unbind from vfio-pci after {max_retries} attempts: {e}"
|
||||
)
|
||||
# Continue with restore attempt anyway
|
||||
break
|
||||
|
||||
if not unbind_successful and get_current_driver(bdf) == "vfio-pci":
|
||||
logger.warning(
|
||||
f"Device {bdf} still bound to vfio-pci, restore may fail"
|
||||
)
|
||||
else:
|
||||
logger.info(
|
||||
f"Device {bdf} not bound to vfio-pci (current: {current_driver or 'none'})"
|
||||
|
|
@ -353,18 +657,85 @@ def restore_original_driver(bdf: str, original_driver: Optional[str]) -> None:
|
|||
if original_driver:
|
||||
# Check if original driver is available
|
||||
driver_path = f"/sys/bus/pci/drivers/{original_driver}"
|
||||
if os.path.exists(driver_path):
|
||||
logger.debug(f"Binding {bdf} back to {original_driver}")
|
||||
run_command(f"echo {bdf} > /sys/bus/pci/drivers/{original_driver}/bind")
|
||||
logger.info(f"Successfully restored {bdf} to {original_driver}")
|
||||
else:
|
||||
if not os.path.exists(driver_path):
|
||||
logger.warning(
|
||||
f"Original driver {original_driver} not available for restore"
|
||||
)
|
||||
print(f"Warning: Original driver {original_driver} not available")
|
||||
return
|
||||
|
||||
logger.debug(f"Binding {bdf} back to {original_driver}")
|
||||
max_retries = 3
|
||||
restore_successful = False
|
||||
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
# Use direct file writing instead of shell redirection
|
||||
bind_path = f"/sys/bus/pci/drivers/{original_driver}/bind"
|
||||
with open(bind_path, "w") as f:
|
||||
f.write(f"{bdf}\n")
|
||||
|
||||
# Verify restore was successful
|
||||
if _wait_for_device_state(bdf, original_driver, max_retries=3):
|
||||
logger.info(f"Successfully restored {bdf} to {original_driver}")
|
||||
print(f"[✓] Device restored to {original_driver} driver")
|
||||
restore_successful = True
|
||||
break
|
||||
elif attempt < max_retries - 1:
|
||||
logger.warning(
|
||||
f"Restore command succeeded but device not bound to {original_driver} (attempt {attempt + 1}), retrying..."
|
||||
)
|
||||
import time
|
||||
|
||||
time.sleep(1)
|
||||
|
||||
except (OSError, IOError) as e:
|
||||
if "Device or resource busy" in str(e):
|
||||
logger.warning(
|
||||
f"Device {bdf} is busy during restore (attempt {attempt + 1})"
|
||||
)
|
||||
if attempt < max_retries - 1:
|
||||
import time
|
||||
|
||||
time.sleep(2)
|
||||
continue
|
||||
elif "No such device" in str(
|
||||
e
|
||||
) or "No such file or directory" in str(e):
|
||||
logger.warning(f"Device {bdf} not found during restore")
|
||||
break
|
||||
elif attempt < max_retries - 1:
|
||||
logger.warning(
|
||||
f"Failed to restore to {original_driver} (attempt {attempt + 1}): {e}, retrying..."
|
||||
)
|
||||
import time
|
||||
|
||||
time.sleep(1)
|
||||
else:
|
||||
logger.warning(
|
||||
f"Failed to restore to {original_driver} after {max_retries} attempts: {e}"
|
||||
)
|
||||
break
|
||||
|
||||
if not restore_successful:
|
||||
final_driver = get_current_driver(bdf)
|
||||
if final_driver == original_driver:
|
||||
logger.info(
|
||||
f"Device {bdf} is bound to {original_driver} despite restore errors"
|
||||
)
|
||||
print(f"[✓] Device is bound to {original_driver} driver")
|
||||
else:
|
||||
logger.warning(
|
||||
f"Failed to restore {bdf} to {original_driver}, current driver: {final_driver or 'none'}"
|
||||
)
|
||||
print(
|
||||
f"Warning: Failed to restore to {original_driver}, current driver: {final_driver or 'none'}"
|
||||
)
|
||||
else:
|
||||
logger.info(f"No original driver to restore for {bdf}")
|
||||
print("[*] No original driver to restore")
|
||||
|
||||
except subprocess.CalledProcessError as e:
|
||||
except (OSError, IOError) as e:
|
||||
logger.warning(f"Failed to restore original driver for {bdf}: {e}")
|
||||
print(f"Warning: Failed to restore original driver: {e}")
|
||||
except Exception as e:
|
||||
|
|
@ -372,10 +743,136 @@ def restore_original_driver(bdf: str, original_driver: Optional[str]) -> None:
|
|||
print(f"Warning: Unexpected error during driver restore: {e}")
|
||||
|
||||
|
||||
def _validate_vfio_device_access(vfio_device: str, bdf: str) -> None:
|
||||
"""Validate VFIO device access and permissions."""
|
||||
# Check if VFIO device exists
|
||||
if not os.path.exists(vfio_device):
|
||||
error_msg = f"VFIO device {vfio_device} not found"
|
||||
logger.error(error_msg)
|
||||
raise RuntimeError(error_msg)
|
||||
|
||||
# Check if /dev/vfio/vfio exists (VFIO container device)
|
||||
vfio_container = "/dev/vfio/vfio"
|
||||
if not os.path.exists(vfio_container):
|
||||
error_msg = f"VFIO container device {vfio_container} not found"
|
||||
logger.error(error_msg)
|
||||
raise RuntimeError(error_msg)
|
||||
|
||||
# Check device permissions
|
||||
try:
|
||||
import stat
|
||||
|
||||
vfio_stat = os.stat(vfio_device)
|
||||
if not (vfio_stat.st_mode & stat.S_IRGRP) or not (
|
||||
vfio_stat.st_mode & stat.S_IWGRP
|
||||
):
|
||||
logger.warning(
|
||||
f"VFIO device {vfio_device} may not have proper group permissions"
|
||||
)
|
||||
except OSError as e:
|
||||
logger.warning(f"Could not check VFIO device permissions: {e}")
|
||||
|
||||
# Verify device is actually bound to vfio-pci
|
||||
current_driver = get_current_driver(bdf)
|
||||
if current_driver != "vfio-pci":
|
||||
error_msg = (
|
||||
f"Device {bdf} not bound to vfio-pci (current: {current_driver or 'none'})"
|
||||
)
|
||||
logger.error(error_msg)
|
||||
raise RuntimeError(error_msg)
|
||||
|
||||
|
||||
def _validate_container_environment() -> None:
|
||||
"""Validate container runtime environment."""
|
||||
# Check if podman is available
|
||||
if shutil.which("podman") is None:
|
||||
error_msg = "Podman not found in PATH. Please install Podman container runtime."
|
||||
logger.error(error_msg)
|
||||
raise RuntimeError(error_msg)
|
||||
|
||||
# Check if container image exists
|
||||
try:
|
||||
result = run_command(
|
||||
"podman images --format '{{.Repository}}:{{.Tag}}' | grep '^pcileech-fw-generator:'",
|
||||
timeout=10,
|
||||
)
|
||||
if not result:
|
||||
# Container image not found, try to build it automatically
|
||||
logger.info(
|
||||
"Container image 'pcileech-fw-generator' not found. Building it now..."
|
||||
)
|
||||
print(
|
||||
"[*] Container image 'pcileech-fw-generator' not found. Building it now..."
|
||||
)
|
||||
|
||||
try:
|
||||
# Use the proper build script which handles the container building correctly
|
||||
build_script_path = "scripts/build_container.sh"
|
||||
if os.path.exists(build_script_path):
|
||||
logger.info("Using build script for container creation")
|
||||
build_result = subprocess.run(
|
||||
f"bash {build_script_path} --tag pcileech-fw-generator:latest",
|
||||
shell=True,
|
||||
check=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
)
|
||||
else:
|
||||
# Fallback to direct container build
|
||||
build_result = subprocess.run(
|
||||
"podman build -t pcileech-fw-generator:latest -f Containerfile .",
|
||||
shell=True,
|
||||
check=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
)
|
||||
logger.info("Container image built successfully")
|
||||
print("[✓] Container image built successfully")
|
||||
except subprocess.CalledProcessError as e:
|
||||
error_msg = f"Failed to build container image automatically: {e.stderr}\nPlease build manually with: make container"
|
||||
logger.error(error_msg)
|
||||
raise RuntimeError(error_msg)
|
||||
except subprocess.CalledProcessError:
|
||||
# If we can't check, try to build anyway
|
||||
logger.info("Could not check container image status. Attempting to build...")
|
||||
print("[*] Could not check container image status. Attempting to build...")
|
||||
|
||||
try:
|
||||
build_script_path = "scripts/build_container.sh"
|
||||
if os.path.exists(build_script_path):
|
||||
logger.info("Using build script for container creation")
|
||||
build_result = subprocess.run(
|
||||
f"bash {build_script_path} --tag pcileech-fw-generator:latest",
|
||||
shell=True,
|
||||
check=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
)
|
||||
else:
|
||||
# Fallback to direct container build
|
||||
build_result = subprocess.run(
|
||||
"podman build -t pcileech-fw-generator:latest -f Containerfile .",
|
||||
shell=True,
|
||||
check=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
)
|
||||
logger.info("Container image built successfully")
|
||||
print("[✓] Container image built successfully")
|
||||
except subprocess.CalledProcessError as e:
|
||||
error_msg = f"Failed to build container image: {e.stderr}\nPlease build manually with: make container"
|
||||
logger.error(error_msg)
|
||||
raise RuntimeError(error_msg)
|
||||
|
||||
|
||||
def run_build_container(
|
||||
bdf: str, board: str, vfio_device: str, args: argparse.Namespace
|
||||
) -> None:
|
||||
"""Run the firmware build in a Podman container"""
|
||||
"""Run the firmware build in a Podman container with enhanced validation and error handling."""
|
||||
if not validate_bdf_format(bdf):
|
||||
raise ValueError(
|
||||
f"Invalid BDF format: {bdf}. Expected format: DDDD:BB:DD.F (e.g., 0000:03:00.0)"
|
||||
|
|
@ -396,17 +893,24 @@ def run_build_container(
|
|||
|
||||
logger.info(f"Starting container build for device {bdf} on board {board}")
|
||||
|
||||
# Ensure output directory exists
|
||||
os.makedirs("output", exist_ok=True)
|
||||
# Validate container environment
|
||||
_validate_container_environment()
|
||||
|
||||
# Validate VFIO device exists
|
||||
if not os.path.exists(vfio_device):
|
||||
error_msg = f"VFIO device {vfio_device} not found"
|
||||
# Ensure output directory exists with proper permissions
|
||||
output_dir = "output"
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
|
||||
# Check output directory permissions
|
||||
if not os.access(output_dir, os.W_OK):
|
||||
error_msg = f"Output directory {output_dir} is not writable"
|
||||
logger.error(error_msg)
|
||||
raise RuntimeError(error_msg)
|
||||
|
||||
# Validate VFIO device access
|
||||
_validate_vfio_device_access(vfio_device, bdf)
|
||||
|
||||
# Build the build.py command with all arguments - use modular build system if available
|
||||
build_cmd_parts = [f"sudo python3 /app/build.py --bdf {bdf} --board {board}"]
|
||||
build_cmd_parts = [f"sudo python3 /app/src/build.py --bdf {bdf} --board {board}"]
|
||||
|
||||
# Add advanced features arguments
|
||||
if args.advanced_sv:
|
||||
|
|
@ -441,7 +945,7 @@ def run_build_container(
|
|||
--device={vfio_device} \
|
||||
--device=/dev/vfio/vfio \
|
||||
-v {os.getcwd()}/output:/app/output \
|
||||
dma-fw \
|
||||
pcileech-fw-generator:latest \
|
||||
{build_cmd}
|
||||
"""
|
||||
).strip()
|
||||
|
|
@ -583,7 +1087,7 @@ def validate_environment() -> None:
|
|||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent / "src"))
|
||||
from src.vivado_utils import find_vivado_installation
|
||||
from src.vivado_utils import find_vivado_installation, get_vivado_search_paths
|
||||
|
||||
vivado_info = find_vivado_installation()
|
||||
if vivado_info:
|
||||
|
|
@ -592,23 +1096,89 @@ def validate_environment() -> None:
|
|||
)
|
||||
print(f"[✓] Vivado {vivado_info['version']} detected")
|
||||
else:
|
||||
logger.warning("Vivado not found. It will be used from the container.")
|
||||
print("[!] Warning: Vivado not found locally. Will use container version.")
|
||||
# Show what paths were checked using the utility function
|
||||
checked_paths = get_vivado_search_paths()
|
||||
error_msg = f"Vivado not found. Checked paths:\n" + "\n".join(
|
||||
f" • {path}" for path in checked_paths
|
||||
)
|
||||
logger.error(error_msg)
|
||||
print(f"[✗] {error_msg}")
|
||||
|
||||
# Allow user to skip
|
||||
try:
|
||||
response = (
|
||||
input("\nWould you like to continue without Vivado? (y/N): ")
|
||||
.strip()
|
||||
.lower()
|
||||
)
|
||||
if response in ["y", "yes"]:
|
||||
print("[!] Continuing without Vivado - some features may not work")
|
||||
logger.warning("User chose to continue without Vivado")
|
||||
else:
|
||||
raise RuntimeError(
|
||||
"Vivado is required. Please install Vivado and try again."
|
||||
)
|
||||
except (KeyboardInterrupt, EOFError):
|
||||
raise RuntimeError(
|
||||
"Vivado is required. Please install Vivado and try again."
|
||||
)
|
||||
except ImportError:
|
||||
logger.warning("Could not import vivado_utils. Skipping Vivado check.")
|
||||
print("[!] Warning: Skipping Vivado check.")
|
||||
error_msg = (
|
||||
"Could not import vivado_utils. Please ensure Vivado is properly installed."
|
||||
)
|
||||
logger.error(error_msg)
|
||||
print(f"[✗] {error_msg}")
|
||||
|
||||
# Allow user to skip
|
||||
try:
|
||||
response = (
|
||||
input("\nWould you like to continue without Vivado? (y/N): ")
|
||||
.strip()
|
||||
.lower()
|
||||
)
|
||||
if response in ["y", "yes"]:
|
||||
print("[!] Continuing without Vivado - some features may not work")
|
||||
logger.warning("User chose to continue without Vivado")
|
||||
else:
|
||||
raise RuntimeError(
|
||||
"Vivado is required. Please install Vivado and try again."
|
||||
)
|
||||
except (KeyboardInterrupt, EOFError):
|
||||
raise RuntimeError(
|
||||
"Vivado is required. Please install Vivado and try again."
|
||||
)
|
||||
|
||||
# Check if container image exists
|
||||
try:
|
||||
result = run_command("podman images dma-fw --format '{{.Repository}}'")
|
||||
if "dma-fw" not in result:
|
||||
result = run_command(
|
||||
"podman images pcileech-fw-generator --format '{{.Repository}}'"
|
||||
)
|
||||
if "pcileech-fw-generator" not in result:
|
||||
# Container image not found, try to build it
|
||||
logger.info("Container image 'dma-fw' not found. Building it now...")
|
||||
print("[*] Container image 'dma-fw' not found. Building it now...")
|
||||
logger.info(
|
||||
"Container image 'pcileech-fw-generator' not found. Building it now..."
|
||||
)
|
||||
print(
|
||||
"[*] Container image 'pcileech-fw-generator' not found. Building it now..."
|
||||
)
|
||||
|
||||
try:
|
||||
# Use the proper build script which handles the container building correctly
|
||||
build_script_path = "scripts/build_container.sh"
|
||||
if os.path.exists(build_script_path):
|
||||
logger.info("Using build script for container creation")
|
||||
build_result = subprocess.run(
|
||||
"podman build -t dma-fw .",
|
||||
f"bash {build_script_path} --tag pcileech-fw-generator:latest",
|
||||
shell=True,
|
||||
check=True,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
)
|
||||
else:
|
||||
# Fallback to direct container build
|
||||
build_result = subprocess.run(
|
||||
"podman build -t pcileech-fw-generator:latest -f Containerfile .",
|
||||
shell=True,
|
||||
check=True,
|
||||
stdout=subprocess.PIPE,
|
||||
|
|
|
|||
649
junit-unit.xml
649
junit-unit.xml
File diff suppressed because one or more lines are too long
|
|
@ -1,11 +1,18 @@
|
|||
#!/bin/bash
|
||||
# Wrapper script to run pcileech-build with sudo while preserving Python path
|
||||
# Wrapper script to run pcileech-generate with sudo while preserving Python path
|
||||
#
|
||||
# generate.py is the correct entry point that orchestrates:
|
||||
# • Device enumeration and selection
|
||||
# • Driver rebinding to vfio-pci
|
||||
# • Container execution with build.py
|
||||
# • Optional firmware flashing
|
||||
|
||||
# Get the path to the installed pcileech-build script
|
||||
PCILEECH_BUILD_PATH=$(which pcileech-build)
|
||||
# Get the path to the installed pcileech-generate script
|
||||
PCILEECH_GENERATE_PATH=$(which pcileech-generate)
|
||||
|
||||
if [ -z "$PCILEECH_BUILD_PATH" ]; then
|
||||
echo "Error: pcileech-build not found in PATH"
|
||||
if [ -z "$PCILEECH_GENERATE_PATH" ]; then
|
||||
echo "Error: pcileech-generate not found in PATH"
|
||||
echo "Make sure pcileechfwgenerator is properly installed: pip install pcileechfwgenerator"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
|
@ -14,5 +21,5 @@ PYTHON_USER_SITE=$(python3 -m site --user-site)
|
|||
PYTHON_SITE_PACKAGES=$(python3 -c "import site; print(':'.join(site.getsitepackages()))")
|
||||
|
||||
# Run with sudo, preserving the PYTHONPATH
|
||||
echo "Running pcileech-build with sudo and preserved Python paths..."
|
||||
sudo PYTHONPATH="$PYTHONPATH:$PYTHON_USER_SITE:$PYTHON_SITE_PACKAGES" "$PCILEECH_BUILD_PATH" "$@"
|
||||
echo "Running pcileech-generate with sudo and preserved Python paths..."
|
||||
sudo PYTHONPATH="$PYTHONPATH:$PYTHON_USER_SITE:$PYTHON_SITE_PACKAGES" "$PCILEECH_GENERATE_PATH" "$@"
|
||||
|
|
@ -80,7 +80,18 @@ pcileech-tui = "src.tui_cli:main"
|
|||
pcileech-build = "src.build_cli:main"
|
||||
|
||||
[tool.setuptools]
|
||||
packages = ["src", "src.tui", "src.tui.core", "src.tui.models", "src.scripts"]
|
||||
packages = [
|
||||
"src",
|
||||
"src.tui",
|
||||
"src.tui.core",
|
||||
"src.tui.models",
|
||||
"src.scripts",
|
||||
"src.build",
|
||||
"src.build.analysis",
|
||||
"src.build.config",
|
||||
"src.build.orchestration",
|
||||
"src.build.generators"
|
||||
]
|
||||
|
||||
[tool.setuptools.dynamic]
|
||||
version = {attr = "src.__version__.__version__"}
|
||||
|
|
|
|||
Loading…
Reference in New Issue