Browse Source
Add incremental data streaming, GIL release, GPL-3.0 license
Add incremental data streaming, GIL release, GPL-3.0 license
- Add IncrementalBuffer with thread-safe SendData/SendInitData callback handling for live waveform plotting during transient simulations - Expose get_incremental_data(), get_incremental_vector_names(), get_incremental_count(), clear_incremental_buffer() to Python - Add py::call_guard<py::gil_scoped_release>() on run(), run_async(), SimRunner::run_now(), SimRunner::run() so Python UI stays responsive - Update license from BSD-3-Clause to GPL-3.0-or-later (required by GPLv3 components: admst, xspice/icm/table) - Fix netlist preprocessor robustness when .end line is missing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>pre-master-46
7 changed files with 434 additions and 10 deletions
-
221INCREMENTAL_DATA_SPEC.md
-
44LICENSE
-
20pyngspice/netlist.py
-
4pyproject.toml
-
24src/bindings/module.cpp
-
96src/cpp/simulator.cpp
-
35src/cpp/simulator.h
@ -0,0 +1,221 @@ |
|||||
|
# Incremental Simulation Data Feature - Implementation Spec |
||||
|
|
||||
|
## Goal |
||||
|
|
||||
|
Enable live waveform plotting in pyTesla by exposing ngspice's `SendData` callback data to Python. During a transient simulation, pyTesla needs to periodically read partial results and update the plot while the simulation is still running. |
||||
|
|
||||
|
## Current State |
||||
|
|
||||
|
The `send_data_callback` is already registered with ngspice in `simulator.cpp` line 71, but the handler `handle_data()` is empty - it discards the data. The `pvecvaluesall` struct that ngspice passes to this callback contains ALL vector values at each simulation timestep. |
||||
|
|
||||
|
The relevant data structures from `sharedspice.h`: |
||||
|
|
||||
|
```cpp |
||||
|
typedef struct vecvalues { |
||||
|
char* name; // Vector name (e.g., "v(top)", "i(l1)") |
||||
|
double creal; // Real value at this timestep |
||||
|
double cimag; // Imaginary value |
||||
|
NG_BOOL is_scale; // True if this is the sweep/time variable |
||||
|
NG_BOOL is_complex; |
||||
|
} vecvalues, *pvecvalues; |
||||
|
|
||||
|
typedef struct vecvaluesall { |
||||
|
int veccount; // Number of vectors |
||||
|
int vecindex; // Current step index |
||||
|
pvecvalues *vecsa; // Array of vector value pointers |
||||
|
} vecvaluesall, *pvecvaluesall; |
||||
|
``` |
||||
|
|
||||
|
## Required Changes |
||||
|
|
||||
|
### 1. Add incremental data buffer to Simulator class (`simulator.h`) |
||||
|
|
||||
|
Add these members to the `Simulator` class: |
||||
|
|
||||
|
```cpp |
||||
|
private: |
||||
|
// Incremental data buffer - written by SendData callback, read by Python |
||||
|
struct IncrementalBuffer { |
||||
|
std::vector<std::string> vector_names; // Set once by SendInitData |
||||
|
std::vector<std::vector<double>> data; // [vector_index][step] = value |
||||
|
size_t read_cursor = 0; // How far Python has read |
||||
|
bool initialized = false; // True after SendInitData sets names |
||||
|
std::mutex mutex; // Separate mutex (callback is hot path) |
||||
|
}; |
||||
|
IncrementalBuffer incr_buffer_; |
||||
|
``` |
||||
|
|
||||
|
### 2. Implement handle_init_data() (`simulator.cpp`) |
||||
|
|
||||
|
When ngspice calls `SendInitData` before simulation starts, capture the vector names: |
||||
|
|
||||
|
```cpp |
||||
|
void Simulator::handle_init_data(pvecinfoall data) { |
||||
|
std::lock_guard<std::mutex> lock(incr_buffer_.mutex); |
||||
|
incr_buffer_.vector_names.clear(); |
||||
|
incr_buffer_.data.clear(); |
||||
|
incr_buffer_.read_cursor = 0; |
||||
|
incr_buffer_.initialized = false; |
||||
|
|
||||
|
if (data && data->veccount > 0) { |
||||
|
for (int i = 0; i < data->veccount; i++) { |
||||
|
if (data->vecs[i] && data->vecs[i]->vecname) { |
||||
|
incr_buffer_.vector_names.push_back(data->vecs[i]->vecname); |
||||
|
} |
||||
|
} |
||||
|
incr_buffer_.data.resize(incr_buffer_.vector_names.size()); |
||||
|
incr_buffer_.initialized = true; |
||||
|
} |
||||
|
} |
||||
|
``` |
||||
|
|
||||
|
### 3. Implement handle_data() (`simulator.cpp`) |
||||
|
|
||||
|
When ngspice calls `SendData` during simulation, append the values: |
||||
|
|
||||
|
```cpp |
||||
|
void Simulator::handle_data(pvecvaluesall data, int count) { |
||||
|
if (!data || !incr_buffer_.initialized) return; |
||||
|
|
||||
|
std::lock_guard<std::mutex> lock(incr_buffer_.mutex); |
||||
|
|
||||
|
for (int i = 0; i < data->veccount && i < (int)incr_buffer_.data.size(); i++) { |
||||
|
if (data->vecsa[i]) { |
||||
|
incr_buffer_.data[i].push_back(data->vecsa[i]->creal); |
||||
|
} |
||||
|
} |
||||
|
} |
||||
|
``` |
||||
|
|
||||
|
### 4. Add Python-accessible methods to Simulator (`simulator.h` / `simulator.cpp`) |
||||
|
|
||||
|
```cpp |
||||
|
// Returns vector names from the incremental buffer |
||||
|
std::vector<std::string> get_incremental_vector_names() const; |
||||
|
|
||||
|
// Returns new data since last call. |
||||
|
// Returns map: vector_name -> vector of new values since last read. |
||||
|
// Advances the read cursor. |
||||
|
std::map<std::string, std::vector<double>> get_incremental_data(); |
||||
|
|
||||
|
// Returns total number of data points buffered so far |
||||
|
size_t get_incremental_count() const; |
||||
|
|
||||
|
// Clears the incremental buffer (call before starting a new simulation) |
||||
|
void clear_incremental_buffer(); |
||||
|
``` |
||||
|
|
||||
|
Implementation of `get_incremental_data()`: |
||||
|
|
||||
|
```cpp |
||||
|
std::map<std::string, std::vector<double>> Simulator::get_incremental_data() { |
||||
|
std::lock_guard<std::mutex> lock(incr_buffer_.mutex); |
||||
|
std::map<std::string, std::vector<double>> result; |
||||
|
|
||||
|
if (!incr_buffer_.initialized || incr_buffer_.data.empty()) { |
||||
|
return result; |
||||
|
} |
||||
|
|
||||
|
size_t total = incr_buffer_.data[0].size(); |
||||
|
if (incr_buffer_.read_cursor >= total) { |
||||
|
return result; // No new data |
||||
|
} |
||||
|
|
||||
|
for (size_t i = 0; i < incr_buffer_.vector_names.size(); i++) { |
||||
|
auto begin = incr_buffer_.data[i].begin() + incr_buffer_.read_cursor; |
||||
|
auto end = incr_buffer_.data[i].end(); |
||||
|
result[incr_buffer_.vector_names[i]] = std::vector<double>(begin, end); |
||||
|
} |
||||
|
|
||||
|
incr_buffer_.read_cursor = total; |
||||
|
return result; |
||||
|
} |
||||
|
``` |
||||
|
|
||||
|
### 5. Expose in pybind11 bindings (`module.cpp`) |
||||
|
|
||||
|
Add to the Simulator class bindings: |
||||
|
|
||||
|
```cpp |
||||
|
.def("get_incremental_vector_names", &ngspice::Simulator::get_incremental_vector_names, |
||||
|
"Get vector names available in the incremental buffer") |
||||
|
|
||||
|
.def("get_incremental_data", &ngspice::Simulator::get_incremental_data, |
||||
|
"Get new simulation data since last call. Returns dict of vector_name -> list of new values.") |
||||
|
|
||||
|
.def("get_incremental_count", &ngspice::Simulator::get_incremental_count, |
||||
|
"Get total number of timesteps buffered so far") |
||||
|
|
||||
|
.def("clear_incremental_buffer", &ngspice::Simulator::clear_incremental_buffer, |
||||
|
"Clear the incremental data buffer") |
||||
|
``` |
||||
|
|
||||
|
### 6. Clear buffer on reset (`simulator.cpp`) |
||||
|
|
||||
|
In the existing `reset()` method, add: |
||||
|
|
||||
|
```cpp |
||||
|
void Simulator::reset() { |
||||
|
// ... existing reset code ... |
||||
|
clear_incremental_buffer(); |
||||
|
} |
||||
|
``` |
||||
|
|
||||
|
## Usage Pattern (Python side - for reference only, don't implement) |
||||
|
|
||||
|
```python |
||||
|
sim = Simulator() |
||||
|
sim.initialize() |
||||
|
sim.load_netlist("circuit.net") |
||||
|
sim.command("set filetype=binary") |
||||
|
sim.clear_incremental_buffer() |
||||
|
sim.run_async() # Non-blocking |
||||
|
|
||||
|
while sim.is_running(): |
||||
|
new_data = sim.get_incremental_data() |
||||
|
if new_data: |
||||
|
# new_data is dict: {"time": [0.1, 0.2, ...], "v(top)": [1.5, 2.3, ...], ...} |
||||
|
update_plot(new_data) |
||||
|
time.sleep(0.1) |
||||
|
|
||||
|
# Final read to get any remaining data |
||||
|
final_data = sim.get_incremental_data() |
||||
|
``` |
||||
|
|
||||
|
## Important Notes |
||||
|
|
||||
|
- The `SendData` callback fires on ngspice's simulation thread, so the buffer mutex must be lightweight (no Python GIL interaction) |
||||
|
- `get_incremental_data()` is called from the Python main thread via pybind11, which handles GIL automatically |
||||
|
- The read cursor pattern avoids copying the entire buffer each poll - only new data is returned |
||||
|
- Don't forget to release the GIL when calling `run_async()` so the background thread can actually run (use `py::call_guard<py::gil_scoped_release>()` in the binding if not already done) |
||||
|
- After implementation, rebuild with: `pip install -e .` from the ngspice repo root |
||||
|
|
||||
|
## Files to Modify |
||||
|
|
||||
|
1. `C:\git\ngspice\src\cpp\simulator.h` - Add IncrementalBuffer struct and new methods |
||||
|
2. `C:\git\ngspice\src\cpp\simulator.cpp` - Implement handle_data(), handle_init_data(), and new methods |
||||
|
3. `C:\git\ngspice\src\bindings\module.cpp` - Add pybind11 bindings for new methods |
||||
|
|
||||
|
## Testing |
||||
|
|
||||
|
After building, verify with: |
||||
|
|
||||
|
```python |
||||
|
from pyngspice._pyngspice import Simulator |
||||
|
sim = Simulator() |
||||
|
sim.initialize() |
||||
|
# Load a simple transient netlist |
||||
|
sim.load_netlist("test.net") |
||||
|
sim.clear_incremental_buffer() |
||||
|
sim.run_async() |
||||
|
import time |
||||
|
while sim.is_running(): |
||||
|
data = sim.get_incremental_data() |
||||
|
if data: |
||||
|
for name, values in data.items(): |
||||
|
print(f" {name}: {len(values)} new points") |
||||
|
time.sleep(0.1) |
||||
|
# Final read |
||||
|
data = sim.get_incremental_data() |
||||
|
print(f"Final: {sum(len(v) for v in data.values()) // max(len(data),1)} total points per vector") |
||||
|
``` |
||||
@ -0,0 +1,44 @@ |
|||||
|
pyngspice - Python bindings for ngspice circuit simulator |
||||
|
========================================================= |
||||
|
|
||||
|
Copyright (C) 2025 pyngspice contributors |
||||
|
|
||||
|
This program is free software: you can redistribute it and/or modify |
||||
|
it under the terms of the GNU General Public License as published by |
||||
|
the Free Software Foundation, either version 3 of the License, or |
||||
|
(at your option) any later version. |
||||
|
|
||||
|
This program is distributed in the hope that it will be useful, |
||||
|
but WITHOUT ANY WARRANTY; without even the implied warranty of |
||||
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
||||
|
GNU General Public License for more details. |
||||
|
|
||||
|
You should have received a copy of the GNU General Public License |
||||
|
along with this program. If not, see <https://www.gnu.org/licenses/>. |
||||
|
|
||||
|
|
||||
|
Third-Party Licenses |
||||
|
==================== |
||||
|
|
||||
|
pyngspice statically links the ngspice circuit simulator, which contains |
||||
|
code under several licenses. See COPYING for the full ngspice license |
||||
|
details. Summary: |
||||
|
|
||||
|
- ngspice core: Modified BSD (3-clause) |
||||
|
Copyright 1985-2018, Regents of the University of California and others |
||||
|
|
||||
|
- XSPICE: Public domain (Georgia Tech Research Corporation) |
||||
|
Except src/xspice/icm/table: GPLv2 or later |
||||
|
|
||||
|
- numparam (src/frontend/numparam): LGPLv2 or later |
||||
|
|
||||
|
- KLU (src/maths/KLU): LGPLv2 |
||||
|
|
||||
|
- OSDI (src/osdi): Mozilla Public License 2.0 |
||||
|
|
||||
|
- cppduals (src/include/cppduals): Mozilla Public License 2.0 |
||||
|
|
||||
|
- admst (src/spicelib/devices/adms/admst): GPLv3 |
||||
|
|
||||
|
GPLv3 was chosen for pyngspice to ensure compatibility with all of |
||||
|
the above component licenses. |
||||
Write
Preview
Loading…
Cancel
Save
Reference in new issue