- C++ 77.6%
- PowerShell 7.1%
- Cuda 5.8%
- JavaScript 5.7%
- CMake 2.2%
- Other 1.5%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
reinitialized
4a73912de9
- Updated README.md to reflect changes in fixture file naming. - Modified generate_reference_fixtures.py to enforce model directory argument. - Changed include directive in test_tokenizer.cpp to use the new fixture header. - Created tokenizer_parity_fixtures.h to store frozen tokenizer test cases and metadata. |
||
| .vscode | ||
| apps | ||
| bench | ||
| cmake | ||
| deployment | ||
| docs | ||
| include/pascalrt | ||
| licenses | ||
| quality | ||
| schema | ||
| scripts | ||
| src | ||
| tests | ||
| third_party | ||
| .gitattributes | ||
| .gitignore | ||
| .gitmodules | ||
| AGENTS.md | ||
| CLAUDE.md | ||
| CMakeLists.txt | ||
| CMakePresets.json | ||
| package-lock.json | ||
| package.json | ||
| README.md | ||
| RESULTS.md | ||
PascalRT + Local Workbench
PascalRT is a native C++20/CUDA inference runtime built for one machine: Windows 11, Core i7-8700, GeForce GTX 1070. All inference, kernel and template code is project-owned — no llama.cpp/GGML, Jinja or cuBLAS in the build.
This monorepo holds two independently runnable products.
| Component | Source | Run / build |
|---|---|---|
| PascalRT runtime | apps/pascalrt-server/, src/, include/ |
CMake + CUDA 12.9, preset pascal-native-release |
| Local Workbench | apps/workbench/ |
Node.js 24+, npm start -- --open |
| Build and packaging tooling | cmake/, scripts/, deployment/ |
PowerShell 7 |
PascalRT owns inference, hardware control, model management and runtime telemetry. Local Workbench owns the web UI, conversations, explicit memory, projects, local tools and Work/Cowork tasks. They are separate processes with separate APIs and separate data ownership.
Served model families: Qwen3.5 dense (4B/9B Q4_K_M and a 27B IQ2_M
route), qwen35moe (Ornith-1.5 and Hermes/Qwen3.6 35B-A3B) and Gemma4 26B-A4B.
Current measured status is in RESULTS.md.
Build
call "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\Common7\Tools\VsDevCmd.bat" -arch=x64 -host_arch=x64
cd /d D:\Projects\PascelRT
cmake --preset pascal-native-release -B C:/PascalRTBuild/local
cmake --build C:/PascalRTBuild/local --parallel 3
ctest --test-dir C:/PascalRTBuild/local --output-on-failure -j 1 -LE "^live($|-)"
Build trees live outside the source tree. -LE "^live($|-)" skips tests that
need a real model and the GPU. The build pins CUDA 12.9 and
CMAKE_CUDA_ARCHITECTURES=61-real (sm_61, no PTX); configure only through
the presets. pascal-native-debug enables full diagnostic logging.
Run
PascalRT serves an OpenAI-compatible API on port 8080 and starts the
Workbench UI on 8090 by default (--web-ui). --no-web-ui runs only the
API. --workbench-port, --workbench-dir and --workbench-data-dir select
the UI port, an independently installed app and its private data store. When
Node.js 24+ is missing, startup downloads and verifies a pinned portable Node
distribution without administrator rights or PATH changes.
Workbench can also run on its own against a managed PascalRT installation:
npm start -- --runtime-key-file "$env:LOCALAPPDATA\PascalRT\config\api-key.txt" --open
Workbench has no production npm dependencies; npm ci installs development
dependencies for npm test. Model files stay under D:\LLMWeights and are
never copied into this tree.
Documentation
| Document | Contents |
|---|---|
AGENTS.md |
Working agreement: layout, constraints, build/test, measurement rules |
RESULTS.md |
The measurement ledger — all performance evidence |
docs/NATIVE-DESKTOP.md |
Runtime architecture, memory budgets, model coverage |
docs/DESKTOP-PACKAGE.md |
Desktop package build and setup |
docs/MODEL-MANAGER.md |
Model registry and import |
docs/LOGGING.md |
Diagnostic log format and --log-level / --log-file |
docs/WEBUI-ACCESS.md |
Workbench access controls and LAN exposure |
docs/THERMAL-POWER-OPERATOR-OPTIONS.md |
Operator-approval thermal and power items |
scripts/README.md |
Harness, benchmark and packaging scripts |
apps/workbench/README.md |
Workbench usage |