NeuralBlocks — Visual Deep Learning Model Builder & Trainer

Drag-and-drop to build neural networks (CNN / RNN / Transformer foundations, 55 node templates), auto-generate trainable PyTorch code, train locally (CPU / CUDA / DirectML hardware adaptive), with a built-in DeepSeek AI assistant (fix / evaluate / build / generate demos).

Architecture design: ARCHITECTURE.md.

< [English] | 简体中文 >

---

✨ Highlights

• 55 node templates — 48 network layers (conv / pool / activation / norm / linear / shape / RNN / Transformer / padding) + 7 data-processing blocks (dataset platform)
• Drag-and-drop canvas — snake auto-layout (adapts to canvas width), one-click arrange, collision avoidance, family-colored node cards, auto-graying of disconnected nodes
• Auto PyTorch codegen — DSL → train/inference scripts; RNN→Linear auto-uses last-step hidden state
• Local training — CPU / CUDA / DirectML adaptive, real-time loss/acc curves + dual progress bars + dark terminal logs
• Dataset platform — Hugging Face official search (fuzzy match + downloads + description), HF / local / agent-collect sources, 7 processing blocks, multi-input
• AI assistant (DeepSeek) — 15 tools (incl. dataset search/download, IO module, generate + self-run demos), multi-session + history context + local persistence, streaming output
• IO module — hand-write or let AI generate load_input / load_dataset / format_output, dry-run validated in a subprocess
• Smart environment setup — one-click rebuild (clear read-only + retry), full China mirrors (pip / torch / Hugging Face), SSL cert self-check, no-Python guidance
• Polished UX — unified design language (rounded corners / shadows / animation), VSCode-style bottom tabs, full-column layout, app menu bar

🚀 Quick Start

# 1. Install frontend deps (first time)
npm install

# 2. Start dev (builds main/preload/renderer, spawns Python backend)
npm run dev

Layout: left template library → drag onto canvas → right inspector/AI panel → bottom training/inference tabs → top menu (File/Edit/View/Window/Help).

First run: a red "Environment not ready · Setup" button appears at top — click it to auto-create the environment with a live progress bar + logs. Env prefers conda python=3.11 (falls back to venv), Tsinghua pip mirror for deps, NVIDIA GPU gets cu128 torch (falls back to DirectML / CPU). All China mirrors, no VPN needed.

🔑 AI Assistant (DeepSeek)

Create config.json in the project root (gitignored, never committed):

{
  "api_key": "sk-your-deepseek-key",
  "base_url": "https://api.deepseek.com",
  "model": "deepseek-chat"
}

AI has 15 tools: get_graph / update_graph / validate / run_training / get_error / evaluate / run_command / get_model_info / generate_demo / run_demo / search_dataset / download_dataset / list_datasets / get_io / update_io. It auto-validates after each change and self-heals on failure; AI changes need your confirmation before being applied.

🧩 Dataset Platform

Open via the "Dataset" button in the training panel:

• Official search — fuzzy-match Hugging Face (repo name / downloads / description), click to fill the source
• Source types — HF download / local image folder / agent-collect script
• Processing blocks — normalize / resize / crop / flip / one-hot / to-tensor / sample (auto-expanded to torchvision transforms)
• "Let AI add a dataset" — describe your need → AI searches → downloads → writes the data flow
• Multi-input — input.inputs maps multiple entries, forward(x_a, x_b, ...)

🧪 Tests

npm run typecheck       # Frontend TS typecheck
npm run test:backend    # Backend pytest (40 cases)
cd python-backend && python -m pytest tests -q

🛠 Commands

Command	Description
npm run dev	Dev mode (hot reload + auto backend)
npm run build	Production build
npm run preview	Preview production build
npm run typecheck	Frontend TS typecheck
npm run test:backend	Backend pytest
npm run dist	Build Windows installer (electron-builder NSIS)

📁 Structure

src/main/          Electron main (spawn backend / port / cleanup / interpreter auto-fix)
src/preload/       contextBridge (expose backend baseUrl)
src/               React frontend (canvas / templates / inspector / training / inference / AI / dataset / settings)
shared/            DSL schema + 55 node templates (authoritative, shared FE/BE)
python-backend/    FastAPI engine (DSL validation / shape inference / codegen / training / hardware / env / AI agent / data flow)

🏗 Architecture

React canvas (renderer)  ⇄  DSL (JSON)  ⇄  Python engine (FastAPI subprocess)
      │                          │                        │
   nodes/edges             single source of truth        validate/codegen/train

• DSL is the single source of truth — frontend, Python engine, and AI assistant all operate on DSL only
• Three-layer boundary — frontend never generates PyTorch, Python never touches UI, AI only operates DSL via tools
• Training progress via SSE; AI chat streams token-by-token

📄 License

MIT (see LICENSE)

🤝 Contributing

PRs welcome. Please pass npm run typecheck + npm run test:backend first. See CONTRIBUTING.md.