Browse Source
Add incremental data streaming, GIL release, GPL-3.0 license
Add incremental data streaming, GIL release, GPL-3.0 license
- Add IncrementalBuffer with thread-safe SendData/SendInitData callback handling for live waveform plotting during transient simulations - Expose get_incremental_data(), get_incremental_vector_names(), get_incremental_count(), clear_incremental_buffer() to Python - Add py::call_guard<py::gil_scoped_release>() on run(), run_async(), SimRunner::run_now(), SimRunner::run() so Python UI stays responsive - Update license from BSD-3-Clause to GPL-3.0-or-later (required by GPLv3 components: admst, xspice/icm/table) - Fix netlist preprocessor robustness when .end line is missing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>pre-master-46
7 changed files with 434 additions and 10 deletions
-
221INCREMENTAL_DATA_SPEC.md
-
44LICENSE
-
20pyngspice/netlist.py
-
4pyproject.toml
-
24src/bindings/module.cpp
-
96src/cpp/simulator.cpp
-
35src/cpp/simulator.h
@ -0,0 +1,221 @@ |
|||
# Incremental Simulation Data Feature - Implementation Spec |
|||
|
|||
## Goal |
|||
|
|||
Enable live waveform plotting in pyTesla by exposing ngspice's `SendData` callback data to Python. During a transient simulation, pyTesla needs to periodically read partial results and update the plot while the simulation is still running. |
|||
|
|||
## Current State |
|||
|
|||
The `send_data_callback` is already registered with ngspice in `simulator.cpp` line 71, but the handler `handle_data()` is empty - it discards the data. The `pvecvaluesall` struct that ngspice passes to this callback contains ALL vector values at each simulation timestep. |
|||
|
|||
The relevant data structures from `sharedspice.h`: |
|||
|
|||
```cpp |
|||
typedef struct vecvalues { |
|||
char* name; // Vector name (e.g., "v(top)", "i(l1)") |
|||
double creal; // Real value at this timestep |
|||
double cimag; // Imaginary value |
|||
NG_BOOL is_scale; // True if this is the sweep/time variable |
|||
NG_BOOL is_complex; |
|||
} vecvalues, *pvecvalues; |
|||
|
|||
typedef struct vecvaluesall { |
|||
int veccount; // Number of vectors |
|||
int vecindex; // Current step index |
|||
pvecvalues *vecsa; // Array of vector value pointers |
|||
} vecvaluesall, *pvecvaluesall; |
|||
``` |
|||
|
|||
## Required Changes |
|||
|
|||
### 1. Add incremental data buffer to Simulator class (`simulator.h`) |
|||
|
|||
Add these members to the `Simulator` class: |
|||
|
|||
```cpp |
|||
private: |
|||
// Incremental data buffer - written by SendData callback, read by Python |
|||
struct IncrementalBuffer { |
|||
std::vector<std::string> vector_names; // Set once by SendInitData |
|||
std::vector<std::vector<double>> data; // [vector_index][step] = value |
|||
size_t read_cursor = 0; // How far Python has read |
|||
bool initialized = false; // True after SendInitData sets names |
|||
std::mutex mutex; // Separate mutex (callback is hot path) |
|||
}; |
|||
IncrementalBuffer incr_buffer_; |
|||
``` |
|||
|
|||
### 2. Implement handle_init_data() (`simulator.cpp`) |
|||
|
|||
When ngspice calls `SendInitData` before simulation starts, capture the vector names: |
|||
|
|||
```cpp |
|||
void Simulator::handle_init_data(pvecinfoall data) { |
|||
std::lock_guard<std::mutex> lock(incr_buffer_.mutex); |
|||
incr_buffer_.vector_names.clear(); |
|||
incr_buffer_.data.clear(); |
|||
incr_buffer_.read_cursor = 0; |
|||
incr_buffer_.initialized = false; |
|||
|
|||
if (data && data->veccount > 0) { |
|||
for (int i = 0; i < data->veccount; i++) { |
|||
if (data->vecs[i] && data->vecs[i]->vecname) { |
|||
incr_buffer_.vector_names.push_back(data->vecs[i]->vecname); |
|||
} |
|||
} |
|||
incr_buffer_.data.resize(incr_buffer_.vector_names.size()); |
|||
incr_buffer_.initialized = true; |
|||
} |
|||
} |
|||
``` |
|||
|
|||
### 3. Implement handle_data() (`simulator.cpp`) |
|||
|
|||
When ngspice calls `SendData` during simulation, append the values: |
|||
|
|||
```cpp |
|||
void Simulator::handle_data(pvecvaluesall data, int count) { |
|||
if (!data || !incr_buffer_.initialized) return; |
|||
|
|||
std::lock_guard<std::mutex> lock(incr_buffer_.mutex); |
|||
|
|||
for (int i = 0; i < data->veccount && i < (int)incr_buffer_.data.size(); i++) { |
|||
if (data->vecsa[i]) { |
|||
incr_buffer_.data[i].push_back(data->vecsa[i]->creal); |
|||
} |
|||
} |
|||
} |
|||
``` |
|||
|
|||
### 4. Add Python-accessible methods to Simulator (`simulator.h` / `simulator.cpp`) |
|||
|
|||
```cpp |
|||
// Returns vector names from the incremental buffer |
|||
std::vector<std::string> get_incremental_vector_names() const; |
|||
|
|||
// Returns new data since last call. |
|||
// Returns map: vector_name -> vector of new values since last read. |
|||
// Advances the read cursor. |
|||
std::map<std::string, std::vector<double>> get_incremental_data(); |
|||
|
|||
// Returns total number of data points buffered so far |
|||
size_t get_incremental_count() const; |
|||
|
|||
// Clears the incremental buffer (call before starting a new simulation) |
|||
void clear_incremental_buffer(); |
|||
``` |
|||
|
|||
Implementation of `get_incremental_data()`: |
|||
|
|||
```cpp |
|||
std::map<std::string, std::vector<double>> Simulator::get_incremental_data() { |
|||
std::lock_guard<std::mutex> lock(incr_buffer_.mutex); |
|||
std::map<std::string, std::vector<double>> result; |
|||
|
|||
if (!incr_buffer_.initialized || incr_buffer_.data.empty()) { |
|||
return result; |
|||
} |
|||
|
|||
size_t total = incr_buffer_.data[0].size(); |
|||
if (incr_buffer_.read_cursor >= total) { |
|||
return result; // No new data |
|||
} |
|||
|
|||
for (size_t i = 0; i < incr_buffer_.vector_names.size(); i++) { |
|||
auto begin = incr_buffer_.data[i].begin() + incr_buffer_.read_cursor; |
|||
auto end = incr_buffer_.data[i].end(); |
|||
result[incr_buffer_.vector_names[i]] = std::vector<double>(begin, end); |
|||
} |
|||
|
|||
incr_buffer_.read_cursor = total; |
|||
return result; |
|||
} |
|||
``` |
|||
|
|||
### 5. Expose in pybind11 bindings (`module.cpp`) |
|||
|
|||
Add to the Simulator class bindings: |
|||
|
|||
```cpp |
|||
.def("get_incremental_vector_names", &ngspice::Simulator::get_incremental_vector_names, |
|||
"Get vector names available in the incremental buffer") |
|||
|
|||
.def("get_incremental_data", &ngspice::Simulator::get_incremental_data, |
|||
"Get new simulation data since last call. Returns dict of vector_name -> list of new values.") |
|||
|
|||
.def("get_incremental_count", &ngspice::Simulator::get_incremental_count, |
|||
"Get total number of timesteps buffered so far") |
|||
|
|||
.def("clear_incremental_buffer", &ngspice::Simulator::clear_incremental_buffer, |
|||
"Clear the incremental data buffer") |
|||
``` |
|||
|
|||
### 6. Clear buffer on reset (`simulator.cpp`) |
|||
|
|||
In the existing `reset()` method, add: |
|||
|
|||
```cpp |
|||
void Simulator::reset() { |
|||
// ... existing reset code ... |
|||
clear_incremental_buffer(); |
|||
} |
|||
``` |
|||
|
|||
## Usage Pattern (Python side - for reference only, don't implement) |
|||
|
|||
```python |
|||
sim = Simulator() |
|||
sim.initialize() |
|||
sim.load_netlist("circuit.net") |
|||
sim.command("set filetype=binary") |
|||
sim.clear_incremental_buffer() |
|||
sim.run_async() # Non-blocking |
|||
|
|||
while sim.is_running(): |
|||
new_data = sim.get_incremental_data() |
|||
if new_data: |
|||
# new_data is dict: {"time": [0.1, 0.2, ...], "v(top)": [1.5, 2.3, ...], ...} |
|||
update_plot(new_data) |
|||
time.sleep(0.1) |
|||
|
|||
# Final read to get any remaining data |
|||
final_data = sim.get_incremental_data() |
|||
``` |
|||
|
|||
## Important Notes |
|||
|
|||
- The `SendData` callback fires on ngspice's simulation thread, so the buffer mutex must be lightweight (no Python GIL interaction) |
|||
- `get_incremental_data()` is called from the Python main thread via pybind11, which handles GIL automatically |
|||
- The read cursor pattern avoids copying the entire buffer each poll - only new data is returned |
|||
- Don't forget to release the GIL when calling `run_async()` so the background thread can actually run (use `py::call_guard<py::gil_scoped_release>()` in the binding if not already done) |
|||
- After implementation, rebuild with: `pip install -e .` from the ngspice repo root |
|||
|
|||
## Files to Modify |
|||
|
|||
1. `C:\git\ngspice\src\cpp\simulator.h` - Add IncrementalBuffer struct and new methods |
|||
2. `C:\git\ngspice\src\cpp\simulator.cpp` - Implement handle_data(), handle_init_data(), and new methods |
|||
3. `C:\git\ngspice\src\bindings\module.cpp` - Add pybind11 bindings for new methods |
|||
|
|||
## Testing |
|||
|
|||
After building, verify with: |
|||
|
|||
```python |
|||
from pyngspice._pyngspice import Simulator |
|||
sim = Simulator() |
|||
sim.initialize() |
|||
# Load a simple transient netlist |
|||
sim.load_netlist("test.net") |
|||
sim.clear_incremental_buffer() |
|||
sim.run_async() |
|||
import time |
|||
while sim.is_running(): |
|||
data = sim.get_incremental_data() |
|||
if data: |
|||
for name, values in data.items(): |
|||
print(f" {name}: {len(values)} new points") |
|||
time.sleep(0.1) |
|||
# Final read |
|||
data = sim.get_incremental_data() |
|||
print(f"Final: {sum(len(v) for v in data.values()) // max(len(data),1)} total points per vector") |
|||
``` |
|||
@ -0,0 +1,44 @@ |
|||
pyngspice - Python bindings for ngspice circuit simulator |
|||
========================================================= |
|||
|
|||
Copyright (C) 2025 pyngspice contributors |
|||
|
|||
This program is free software: you can redistribute it and/or modify |
|||
it under the terms of the GNU General Public License as published by |
|||
the Free Software Foundation, either version 3 of the License, or |
|||
(at your option) any later version. |
|||
|
|||
This program is distributed in the hope that it will be useful, |
|||
but WITHOUT ANY WARRANTY; without even the implied warranty of |
|||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
|||
GNU General Public License for more details. |
|||
|
|||
You should have received a copy of the GNU General Public License |
|||
along with this program. If not, see <https://www.gnu.org/licenses/>. |
|||
|
|||
|
|||
Third-Party Licenses |
|||
==================== |
|||
|
|||
pyngspice statically links the ngspice circuit simulator, which contains |
|||
code under several licenses. See COPYING for the full ngspice license |
|||
details. Summary: |
|||
|
|||
- ngspice core: Modified BSD (3-clause) |
|||
Copyright 1985-2018, Regents of the University of California and others |
|||
|
|||
- XSPICE: Public domain (Georgia Tech Research Corporation) |
|||
Except src/xspice/icm/table: GPLv2 or later |
|||
|
|||
- numparam (src/frontend/numparam): LGPLv2 or later |
|||
|
|||
- KLU (src/maths/KLU): LGPLv2 |
|||
|
|||
- OSDI (src/osdi): Mozilla Public License 2.0 |
|||
|
|||
- cppduals (src/include/cppduals): Mozilla Public License 2.0 |
|||
|
|||
- admst (src/spicelib/devices/adms/admst): GPLv3 |
|||
|
|||
GPLv3 was chosen for pyngspice to ensure compatibility with all of |
|||
the above component licenses. |
|||
Write
Preview
Loading…
Cancel
Save
Reference in new issue