SOEN Examples & Tutorials

Get started quickly with these practical examples covering common SOEN use cases. Each example includes complete YAML configurations and explains the key concepts.

Running Examples

All examples can be run using:

python -m soen_sim_v2.training.examples.run_trial --config path/to/training_config.yaml

For complete setup instructions, see the GitHub repository.

Physical Modeling Example

This example shows how to configure a SOEN model with realistic superconducting parameters:

description: "Physical superconducting device simulation"
seed: 123

training:
  batch_size: 64
  max_epochs: 100
  precision: "32-true"
  
  optimizer:
    name: "adamw"
    lr: 0.0005
    kwargs:
      weight_decay: 1e-5

  losses:
    - name: cross_entropy
      weight: 1.0
    - name: gap_loss
      weight: 0.2
      params:
        margin: 0.1

data:
  data_path: "data/physical_timeseries.h5"
  num_classes: 5
  target_seq_len: 200
  input_encoding: "raw"

model:
  base_model_path: "models/physical_soen.pth"
  use_classifier: true
  use_batch_norm: true
  dt: 195.3125        # Physical time step
  dt_learnable: true  # Allow optimization of time step
  
  time_pooling:
    name: "mean_last_n"
    params: 
      n: 20
      scale: 1.0

# Enable physical parameter tracking
logging:
  project_name: "Physical_SOEN"
  track_learnable_params: true
  log_gradients: true
  metrics:
    - "accuracy" 
    - "perplexity"
    - "power_consumption"

callbacks:
  lr_scheduler:
    type: "cosine"
    max_lr: 0.001
    min_lr: 1e-7
    warmup_epochs: 10
    cycle_epochs: 80
    
  early_stopping:
    monitor: "val_loss"
    patience: 15
    mode: "min"

Key features:

• Learnable time step - Optimizes physical dynamics
• Gap loss regularization - Improves classification margins
• Physical time pooling - Averages over final timesteps
• Parameter tracking - Logs gradients and learnable parameters
• Cosine annealing - Advanced learning rate schedule

Large-Scale Training Configuration

For large models and datasets with advanced optimization:

description: "Large-scale SOEN training with advanced features"
seed: 2024

training:
  batch_size: 256
  max_epochs: 500
  
  # Hardware optimization
  accelerator: "gpu"
  precision: "16-mixed"
  devices: [0, 1, 2, 3]    # Multi-GPU training
  num_workers: 8
  
  # Gradient settings
  accumulate_grad_batches: 4
  gradient_clip_val: 1.0
  gradient_clip_algorithm: "norm"
  
  # Checkpointing
  checkpoint_every_n_epochs: 5
  checkpoint_save_top_k: 5
  save_soen_core_in_checkpoint: true
  
  optimizer:
    name: "adamw" 
    lr: 0.001
    kwargs:
      weight_decay: 0.01
      betas: [0.9, 0.999]
      
  losses:
    - name: cross_entropy
      weight: 1.0
    - name: regularization_loss
      weight: 0.1

data:
  data_path: "data/large_dataset.h5"
  cache_data: false        # Too large to cache
  num_classes: 1000
  val_split: 0.1
  test_split: 0.05
  target_seq_len: 1024

model:
  base_model_path: "models/large_soen.pth"
  use_classifier: true
  use_batch_norm: true
  dt: 78.125
  
  time_pooling:
    name: "max"
    params: 
      scale: 0.5

logging:
  project_name: "LargeScale_SOEN"
  group_name: "Production_Models"
  
  # Advanced logging
  log_freq: 100
  log_batch_metrics: true
  track_learnable_params: false  # Too expensive for large models
  
  # Upload to cloud
  upload_logs_and_checkpoints: true
  s3_upload_url: "s3://soen-experiments/"

callbacks:
  lr_scheduler:
    type: "cosine"
    max_lr: 0.002
    min_lr: 1e-8
    warmup_epochs: 50
    cycle_epochs: 400
    
  early_stopping:
    monitor: "val_accuracy"
    patience: 20
    mode: "max"

Large-scale features:

• Multi-GPU training - Distributed across multiple GPUs
• Mixed precision - 16-bit training for speed
• Gradient accumulation - Simulate larger batch sizes
• Cloud logging - Upload results to S3
• Extended schedules - Long warmup and cosine cycles

Model Architecture Examples

Simple Feed-Forward SOEN

# layers.yaml - Define your model architecture
layers:
  - layer_id: 0
    layer_type: "Input"
    params:
      dim: 784        # MNIST input size
      
  - layer_id: 1
    layer_type: "SingleDendrite"
    params:
      dim: 128
      solver: "FE"
      gamma_plus: 1.0
      gamma_minus: 0.1
      
  - layer_id: 2
    layer_type: "SingleDendrite" 
    params:
      dim: 64
      solver: "FE"
      
  - layer_id: 3
    layer_type: "Classifier"
    params:
      dim: 10         # 10 classes

connections:
  - from_layer: 0
    to_layer: 1
    connection_type: "dense"
    
  - from_layer: 1
    to_layer: 2
    connection_type: "dense"
    params:
      init_scale: 0.1
      
  - from_layer: 2
    to_layer: 3
    connection_type: "dense"

Hybrid SOEN-RNN Architecture

layers:
  - layer_id: 0
    layer_type: "Input"
    params:
      dim: 100
      
  - layer_id: 1
    layer_type: "SingleDendrite"
    params:
      dim: 256
      solver: "PS"      # Parallel scan for speed
      track_power: true
      
  - layer_id: 2
    layer_type: "MinGRU"
    params:
      dim: 128
      
  - layer_id: 3
    layer_type: "DoubleDendrite1"
    params:
      dim: 64
      solver: "FE"
      physical_units: true
      
  - layer_id: 4
    layer_type: "Classifier"
    params:
      dim: 5

connections:
  - from_layer: 0
    to_layer: 1
    connection_type: "dense"
    
  # Add recurrent connection  
  - from_layer: 1
    to_layer: 1
    connection_type: "sparse"
    params:
      sparsity: 0.1
      
  - from_layer: 1
    to_layer: 2
    connection_type: "dense"
    
  - from_layer: 2
    to_layer: 3
    connection_type: "dense"
    
  - from_layer: 3
    to_layer: 4
    connection_type: "dense"

Key architectural concepts:

• Layer mixing - Combine physical and virtual layers
• Recurrent connections - Internal connectivity for dynamics
• Solver selection - Choose appropriate integration method
• Power tracking - Monitor energy consumption