Config

Configuration management for TorchLingo.

The Config class provides a centralized way to manage hyperparameters, file paths, and feature toggles. All TorchLingo functions accept an optional config parameter for customization.

Quick Start

from torchlingo.config import Config, get_default_config

# Use defaults
config = get_default_config()

# Customize
config = Config(
    batch_size=64,
    learning_rate=1e-4,
    d_model=512,
)

# Pass to functions
dataset = NMTDataset("data.tsv", config=config)
model = SimpleTransformer(..., config=config)

API Reference

Config

Config(base_dir: Optional[Path] = None, data_dir: Optional[Path] = None, checkpoint_dir: Optional[Path] = None, output_dir: Optional[Path] = None, data_format: str = 'tsv', src_col: str = 'src', tgt_col: str = 'tgt', src_tok_col: str = 'src_tokenized', tgt_tok_col: str = 'tgt_tokenized', train_file: Optional[Path] = None, val_file: Optional[Path] = None, test_file: Optional[Path] = None, raw_data_file: Optional[Path] = None, use_sentencepiece: bool = False, sentencepiece_model_prefix: Optional[str] = None, sentencepiece_model: Optional[str] = None, sentencepiece_vocab: Optional[str] = None, sentencepiece_src_model_prefix: Optional[str] = None, sentencepiece_tgt_model_prefix: Optional[str] = None, sentencepiece_src_model: Optional[str] = None, sentencepiece_tgt_model: Optional[str] = None, sentencepiece_src_vocab: Optional[str] = None, sentencepiece_tgt_vocab: Optional[str] = None, vocab_size: int = 32000, min_freq: int = 2, sp_model_type: str = 'bpe', sp_character_coverage: float = 1.0, sp_normalization_rule_name: str = 'nmt_nfkc', back_trans_src: Optional[Path] = None, back_trans_tgt: Optional[Path] = None, combined_train_src: Optional[Path] = None, combined_train_tgt: Optional[Path] = None, reverse_model_checkpoint: Optional[Path] = None, multi_train_file: Optional[Path] = None, multi_val_file: Optional[Path] = None, test_en_x_file: Optional[Path] = None, test_x_en_file: Optional[Path] = None, lang_tag_en_to_x: str = '<2X>', lang_tag_x_to_en: str = '<2E>', d_model: int = 512, n_heads: int = 8, num_encoder_layers: int = 6, num_decoder_layers: int = 6, d_ff: int = 2048, dropout: float = 0.1, max_seq_length: int = 512, lstm_emb_dim: int = 256, lstm_hidden_dim: int = 512, lstm_num_layers: int = 2, lstm_dropout: float = 0.2, use_packed_projection: bool = False, use_scaled_dot_product_attention: bool = False, tie_embeddings: bool = False, use_compile: bool = False, pad_token: str = '<pad>', unk_token: str = '<unk>', sos_token: str = '<sos>', eos_token: str = '<eos>', pad_idx: int = 0, unk_idx: int = 1, sos_idx: int = 2, eos_idx: int = 3, batch_size: int = 64, num_steps: int = 100000, step_limit: Optional[int] = None, learning_rate: float = 0.0001, adam_betas: tuple = (0.9, 0.98), adam_eps: float = 1e-09, weight_decay: float = 0.0001, warmup_steps: int = 4000, scheduler_type: str = 'cosine', scheduler_patience: int = 3, label_smoothing: float = 0.1, use_bucketing: bool = False, bucket_boundaries: Optional[list] = None, grad_clip: float = 1.0, val_interval: int = 1000, save_interval: int = 5000, log_interval: int = 100, patience: int = 10, checkpoint_path: Optional[Path] = None, last_checkpoint_path: Optional[Path] = None, beam_size: int = 5, max_decode_length: int = 200, length_penalty: float = 0.6, use_greedy: bool = False, output_translations: Optional[Path] = None, output_scores: Optional[Path] = None, device: Optional[str] = None, num_workers: int = 4, seed: int = 42, train_ratio: float = 0.8, val_ratio: float = 0.1, use_tensorboard: bool = False, tensorboard_dir: Optional[Path] = None, experiment_name: str = 'baseline', src_lang: str = 'eng', tgt_lang: str = 'deu')

Instantiable configuration container for TorchLingo training/inference.

This class allows you to create multiple independent config instances, each with its own hyperparameters and settings. Pass a Config instance to train(), evaluate(), or translate() functions.

Note

All attributes correspond to the configuration parameters documented above.

Examples:

>>> # Default configuration
>>> cfg = Config()

>>> # Custom configuration with specific hyperparameters
>>> cfg = Config(batch_size=32, learning_rate=1e-5, beam_size=3)

>>> # Modify on the fly
>>> cfg.batch_size = 128
>>> cfg.dropout = 0.2

>>> # Deep copy for experiments
>>> cfg_exp1 = cfg.copy()
>>> cfg_exp1.learning_rate = 5e-5
>>> # cfg and cfg_exp1 are independent

All parameters are optional. If not provided, defaults from the module-level constants above are used.

Source code in src/torchlingo/config.py

def __init__(
    self,
    # Base directories
    base_dir: Optional[Path] = None,
    data_dir: Optional[Path] = None,
    checkpoint_dir: Optional[Path] = None,
    output_dir: Optional[Path] = None,
    # Data and file settings
    data_format: str = "tsv",
    src_col: str = "src",
    tgt_col: str = "tgt",
    src_tok_col: str = "src_tokenized",
    tgt_tok_col: str = "tgt_tokenized",
    train_file: Optional[Path] = None,
    val_file: Optional[Path] = None,
    test_file: Optional[Path] = None,
    raw_data_file: Optional[Path] = None,
    # Tokenization and vocabulary
    use_sentencepiece: bool = False,
    sentencepiece_model_prefix: Optional[str] = None,
    sentencepiece_model: Optional[str] = None,
    sentencepiece_vocab: Optional[str] = None,
    sentencepiece_src_model_prefix: Optional[str] = None,
    sentencepiece_tgt_model_prefix: Optional[str] = None,
    sentencepiece_src_model: Optional[str] = None,
    sentencepiece_tgt_model: Optional[str] = None,
    sentencepiece_src_vocab: Optional[str] = None,
    sentencepiece_tgt_vocab: Optional[str] = None,
    vocab_size: int = 32000,
    min_freq: int = 2,
    sp_model_type: str = "bpe",
    sp_character_coverage: float = 1.0,
    sp_normalization_rule_name: str = "nmt_nfkc",
    # Back-translation and multilingual
    back_trans_src: Optional[Path] = None,
    back_trans_tgt: Optional[Path] = None,
    combined_train_src: Optional[Path] = None,
    combined_train_tgt: Optional[Path] = None,
    reverse_model_checkpoint: Optional[Path] = None,
    multi_train_file: Optional[Path] = None,
    multi_val_file: Optional[Path] = None,
    test_en_x_file: Optional[Path] = None,
    test_x_en_file: Optional[Path] = None,
    lang_tag_en_to_x: str = "<2X>",
    lang_tag_x_to_en: str = "<2E>",
    # Model architecture
    d_model: int = 512,
    n_heads: int = 8,
    num_encoder_layers: int = 6,
    num_decoder_layers: int = 6,
    d_ff: int = 2048,
    dropout: float = 0.1,
    max_seq_length: int = 512,
    # LSTM model hyperparameters
    lstm_emb_dim: int = 256,
    lstm_hidden_dim: int = 512,
    lstm_num_layers: int = 2,
    lstm_dropout: float = 0.2,
    # Experimental toggles
    use_packed_projection: bool = False,
    use_scaled_dot_product_attention: bool = False,
    tie_embeddings: bool = False,
    use_compile: bool = False,
    # Special tokens and indices
    pad_token: str = "<pad>",
    unk_token: str = "<unk>",
    sos_token: str = "<sos>",
    eos_token: str = "<eos>",
    pad_idx: int = 0,
    unk_idx: int = 1,
    sos_idx: int = 2,
    eos_idx: int = 3,
    # Training hyperparameters
    batch_size: int = 64,
    num_steps: int = 100000,
    step_limit: Optional[int] = None,
    learning_rate: float = 0.0001,
    adam_betas: tuple = (0.9, 0.98),
    adam_eps: float = 1e-9,
    weight_decay: float = 0.0001,
    warmup_steps: int = 4000,
    scheduler_type: str = "cosine",
    scheduler_patience: int = 3,
    label_smoothing: float = 0.1,
    use_bucketing: bool = False,
    bucket_boundaries: Optional[list] = None,
    grad_clip: float = 1.0,
    val_interval: int = 1000,
    save_interval: int = 5000,
    log_interval: int = 100,
    patience: int = 10,
    checkpoint_path: Optional[Path] = None,
    last_checkpoint_path: Optional[Path] = None,
    # Decoding and inference
    beam_size: int = 5,
    max_decode_length: int = 200,
    length_penalty: float = 0.6,
    use_greedy: bool = False,
    output_translations: Optional[Path] = None,
    output_scores: Optional[Path] = None,
    # Device and reproducibility
    device: Optional[str] = None,
    num_workers: int = 4,
    seed: int = 42,
    # Data splitting
    train_ratio: float = 0.8,
    val_ratio: float = 0.1,
    # Experiment tracking
    use_tensorboard: bool = False,
    tensorboard_dir: Optional[Path] = None,
    experiment_name: str = "baseline",
    src_lang: str = "eng",
    tgt_lang: str = "deu",
):
    """Initialize a Config instance with given parameters.

    All parameters are optional. If not provided, defaults from the
    module-level constants above are used.
    """
    # Base directories. Created lazily by code that writes to them, so
    # constructing a Config never touches the filesystem.
    self.base_dir = base_dir or BASE_DIR
    self.data_dir = data_dir or (self.base_dir / "data")
    self.checkpoint_dir = checkpoint_dir or (self.base_dir / "checkpoints")
    self.output_dir = output_dir or (self.base_dir / "outputs")

    # Data and file settings
    self.data_format = data_format
    self.src_col = src_col
    self.tgt_col = tgt_col
    self.src_tok_col = src_tok_col
    self.tgt_tok_col = tgt_tok_col
    self.train_file = train_file or (self.data_dir / f"train.{self.data_format}")
    self.val_file = val_file or (self.data_dir / f"val.{self.data_format}")
    self.test_file = test_file or (self.data_dir / f"test.{self.data_format}")
    self.raw_data_file = raw_data_file or (self.data_dir / "raw_data.tsv")
    self.src_lang = src_lang
    self.tgt_lang = tgt_lang

    # Tokenization and vocabulary
    self.use_sentencepiece = use_sentencepiece
    self.sentencepiece_model_prefix = sentencepiece_model_prefix or str(
        self.data_dir / "sp_model"
    )
    self.sentencepiece_model = sentencepiece_model or (
        self.sentencepiece_model_prefix + ".model"
    )
    self.sentencepiece_vocab = sentencepiece_vocab or (
        self.sentencepiece_model_prefix + ".vocab"
    )
    self.sentencepiece_src_model_prefix = (
        sentencepiece_src_model_prefix or self.sentencepiece_model_prefix
    )
    self.sentencepiece_tgt_model_prefix = (
        sentencepiece_tgt_model_prefix or self.sentencepiece_model_prefix
    )
    self.sentencepiece_src_model = sentencepiece_src_model or (
        self.sentencepiece_src_model_prefix + ".model"
    )
    self.sentencepiece_tgt_model = sentencepiece_tgt_model or (
        self.sentencepiece_tgt_model_prefix + ".model"
    )
    self.sentencepiece_src_vocab = sentencepiece_src_vocab or (
        self.sentencepiece_src_model_prefix + ".vocab"
    )
    self.sentencepiece_tgt_vocab = sentencepiece_tgt_vocab or (
        self.sentencepiece_tgt_model_prefix + ".vocab"
    )
    self.vocab_size = vocab_size
    self.min_freq = min_freq
    self.sp_model_type = sp_model_type
    self.sp_character_coverage = sp_character_coverage
    self.sp_normalization_rule_name = sp_normalization_rule_name

    # Back-translation and multilingual
    self.back_trans_src = back_trans_src or (self.data_dir / "train.back_trans.en")
    self.back_trans_tgt = back_trans_tgt or (self.data_dir / "train.back_trans.x")
    self.combined_train_src = combined_train_src or (
        self.data_dir / "train.combined.en"
    )
    self.combined_train_tgt = combined_train_tgt or (
        self.data_dir / "train.combined.x"
    )
    self.reverse_model_checkpoint = reverse_model_checkpoint or (
        self.checkpoint_dir / "reverse_model_best.pt"
    )
    self.multi_train_file = multi_train_file or (
        self.data_dir / f"train.multi.{self.data_format}"
    )
    self.multi_val_file = multi_val_file or (
        self.data_dir / f"val.multi.{self.data_format}"
    )
    self.test_en_x_file = test_en_x_file or (
        self.data_dir / f"test.en-x.{self.data_format}"
    )
    self.test_x_en_file = test_x_en_file or (
        self.data_dir / f"test.x-en.{self.data_format}"
    )
    self.lang_tag_en_to_x = lang_tag_en_to_x
    self.lang_tag_x_to_en = lang_tag_x_to_en

    # Model architecture
    self.d_model = d_model
    self.n_heads = n_heads
    self.num_encoder_layers = num_encoder_layers
    self.num_decoder_layers = num_decoder_layers
    self.d_ff = d_ff
    self.dropout = dropout
    self.max_seq_length = max_seq_length

    # LSTM model hyperparameters
    self.lstm_emb_dim = lstm_emb_dim
    self.lstm_hidden_dim = lstm_hidden_dim
    self.lstm_num_layers = lstm_num_layers
    self.lstm_dropout = lstm_dropout

    # Experimental toggles
    self.use_packed_projection = use_packed_projection
    self.use_scaled_dot_product_attention = use_scaled_dot_product_attention
    self.tie_embeddings = tie_embeddings
    self.use_compile = use_compile

    # Special tokens and indices
    self.pad_token = pad_token
    self.unk_token = unk_token
    self.sos_token = sos_token
    self.eos_token = eos_token
    self.pad_idx = pad_idx
    self.unk_idx = unk_idx
    self.sos_idx = sos_idx
    self.eos_idx = eos_idx

    # Training hyperparameters
    self.batch_size = batch_size
    self.num_steps = num_steps
    # Optional global step limit (None means no limit)
    self.step_limit = step_limit
    self.learning_rate = learning_rate
    self.adam_betas = adam_betas
    self.adam_eps = adam_eps
    self.weight_decay = weight_decay
    self.warmup_steps = warmup_steps
    self.scheduler_type = scheduler_type
    self.scheduler_patience = scheduler_patience
    self.label_smoothing = label_smoothing
    self.use_bucketing = use_bucketing
    self.bucket_boundaries = bucket_boundaries
    self.grad_clip = grad_clip
    self.val_interval = val_interval
    self.save_interval = save_interval
    self.log_interval = log_interval
    self.patience = patience
    self.checkpoint_path = checkpoint_path or (
        self.checkpoint_dir / "model_best.pt"
    )
    self.last_checkpoint_path = last_checkpoint_path or (
        self.checkpoint_dir / "model_last.pt"
    )

    # Decoding and inference
    self.beam_size = beam_size
    self.max_decode_length = max_decode_length
    self.length_penalty = length_penalty
    self.use_greedy = use_greedy
    self.output_translations = output_translations or (
        self.output_dir / "translations.txt"
    )
    self.output_scores = output_scores or (
        self.output_dir / "translation_scores.txt"
    )

    # Device and reproducibility
    self.device = device or ("cuda" if torch.cuda.is_available() else "cpu")
    self.num_workers = num_workers
    self.seed = seed

    # Data splitting
    self.train_ratio = train_ratio
    self.val_ratio = val_ratio

    # Experiment tracking
    self.use_tensorboard = use_tensorboard
    self.tensorboard_dir = tensorboard_dir or (self.base_dir / "runs")
    self.experiment_name = experiment_name

to_dict

to_dict() -> Dict[str, Any]

Convert config to a dictionary for logging and serialization.

Returns:

Type	Description
Dict[str, Any]	Dictionary with all config attributes (excluding private attributes).

Source code in src/torchlingo/config.py

def to_dict(self) -> Dict[str, Any]:
    """Convert config to a dictionary for logging and serialization.

    Returns:
        Dictionary with all config attributes (excluding private attributes).
    """
    return {k: v for k, v in self.__dict__.items() if not k.startswith("_")}

get_default_config

get_default_config() -> Config

Return a new Config object initialized with module-level defaults.

For experiments that require isolated configs, this function now returns a fresh instance each call, seeded from the current module-level values.

Returns:

Type	Description
Config	A new Config instance.

Source code in src/torchlingo/config.py

def get_default_config() -> Config:
    """Return a new Config object initialized with module-level defaults.

    For experiments that require isolated configs, this function now returns
    a fresh instance each call, seeded from the current module-level values.

    Returns:
        A new Config instance.
    """
    return _default_config.copy()

Configuration Categories

Directory Paths

Parameter	Default	Description
data_dir	src/data	Data files directory
checkpoint_dir	src/checkpoints	Model checkpoints
output_dir	src/outputs	Generated outputs

Data Settings

Parameter	Default	Description
data_format	"tsv"	File format: tsv, csv, parquet, json, txt
src_col	"src"	Source column name
tgt_col	"tgt"	Target column name

Vocabulary Settings

Parameter	Default	Description
min_freq	2	Minimum token frequency
vocab_size	32000	Target vocab size (SentencePiece)
use_sentencepiece	False	Use subword tokenization

Special Tokens

Parameter	Default	Description
pad_token	"<pad>"	Padding token
unk_token	"<unk>"	Unknown token
sos_token	"<sos>"	Start of sequence
eos_token	"<eos>"	End of sequence
pad_idx	0	Padding index
unk_idx	1	Unknown index
sos_idx	2	SOS index
eos_idx	3	EOS index

Model Architecture

Parameter	Default	Description
d_model	512	Transformer hidden dimension
n_heads	8	Attention heads
num_encoder_layers	6	Encoder depth
num_decoder_layers	6	Decoder depth
d_ff	2048	Feed-forward dimension
dropout	0.1	Dropout rate

LSTM Settings

Parameter	Default	Description
lstm_emb_dim	256	Embedding dimension
lstm_hidden_dim	512	Hidden dimension
lstm_num_layers	2	Number of layers
lstm_dropout	0.1	Dropout rate

Training Settings

Parameter	Default	Description
batch_size	64	Batch size
learning_rate	1e-4	Learning rate
max_seq_length	128	Maximum sequence length
num_epochs	10	Training epochs
scheduler_type	"cosine"	LR scheduler: "cosine", "plateau", "transformer", "noam", or "none"
scheduler_patience	3	Validations without improvement before plateau scheduler halves the LR
warmup_steps	4000	Warmup steps for cosine, transformer, and noam schedulers

Experiment Tracking (TensorBoard)

Parameter	Default	Description
use_tensorboard	False	Enable TensorBoard logging
tensorboard_dir	./runs	Directory for TensorBoard event files
experiment_name	"baseline"	Experiment name (becomes subdirectory in tensorboard_dir)
log_interval	100	Steps between logging metrics
val_interval	1000	Steps between validation checks
save_interval	5000	Steps between checkpoint saves

Examples

Creating Custom Configs

# Small model for testing
test_config = Config(
    d_model=64,
    n_heads=2,
    num_encoder_layers=1,
    num_decoder_layers=1,
    batch_size=8,
)

# Production model
prod_config = Config(
    d_model=512,
    n_heads=8,
    num_encoder_layers=6,
    num_decoder_layers=6,
    batch_size=64,
    learning_rate=5e-5,
)

Cloning and Modifying

base = get_default_config()
experiment = base.clone()
experiment.learning_rate = 1e-5
experiment.batch_size = 128

Saving and Loading

import json

# Save to JSON
config = Config(batch_size=32)
with open("config.json", "w") as f:
    json.dump(config.to_dict(), f)

# Load from JSON
with open("config.json", "r") as f:
    config = Config.from_dict(json.load(f))

Module Constants

For maximum compatibility, module-level constants are also available:

from torchlingo import config

print(config.BATCH_SIZE)      # Default batch size
print(config.DATA_DIR)        # Data directory path
print(config.PAD_IDX)         # Padding token index

Prefer Config Objects

Module constants are read-only. For customization, always use Config objects.