Alignment Checks¶

Two cheap checks for whether a parallel corpus is actually parallel, and a length-based aligner for repairing one that is only slightly wrong.

Overview¶

A parallel corpus can be structurally perfect and still be worthless: correct columns, no blank rows, real sentences on both sides, and yet row n of the source is not a translation of row n of the target.

That is not hypothetical. The corpus shipped with this library was once misaligned in exactly that way, and nothing in the repository caught it. A student training on it would have watched the loss refuse to fall with no way to tell bad data from a mistake of their own.

This module is the check that was missing.

Check	Idea	Parallel text	Scrambled
`length_correlation`	A translation is about as long as its source	~0.97	~0.09
`anchor_agreement`	Names and numbers survive translation	~0.41	~0.06

Neither is proof, and each has a blind spot the other covers, so run both. See Data Pipeline for the reasoning and the measured numbers.

API Reference¶

diagnose_alignment ¶

diagnose_alignment(frame: DataFrame, sample: int = DEFAULT_SAMPLE, src_col: str = 'src', tgt_col: str = 'tgt') -> AlignmentReport

Run both alignment checks on a parallel corpus.

Parameters:

Name	Type	Description	Default
`frame`	`DataFrame`	Corpus with source and target columns.	required
`sample`	`int`	Maximum rows to score for anchor agreement.	`DEFAULT_SAMPLE`
`src_col`	`str`	Source column name.	`'src'`
`tgt_col`	`str`	Target column name.	`'tgt'`

Returns:

Name	Type	Description
`AlignmentReport`	`AlignmentReport`	Both measurements, plus how much was scorable.

Example

Two rows at minimum: a correlation needs something to correlate, and a single pair has no length variation to measure.

import pandas as pd frame = pd.DataFrame( ... { ... "src": ["Maria arrived in 1999.", "She spoke briefly."], ... "tgt": ["Maria llegó en 1999.", "Ella habló brevemente."], ... } ... ) diagnose_alignment(frame).anchor_agreement 1.0

Source code in src/torchlingo/preprocessing/alignment.py

def diagnose_alignment(
    frame: pd.DataFrame,
    sample: int = DEFAULT_SAMPLE,
    src_col: str = "src",
    tgt_col: str = "tgt",
) -> AlignmentReport:
    """Run both alignment checks on a parallel corpus.

    Args:
        frame (pd.DataFrame): Corpus with source and target columns.
        sample (int): Maximum rows to score for anchor agreement.
        src_col (str): Source column name.
        tgt_col (str): Target column name.

    Returns:
        AlignmentReport: Both measurements, plus how much was scorable.

    Example:
        Two rows at minimum: a correlation needs something to correlate, and a
        single pair has no length variation to measure.

        >>> import pandas as pd
        >>> frame = pd.DataFrame(
        ...     {
        ...         "src": ["Maria arrived in 1999.", "She spoke briefly."],
        ...         "tgt": ["Maria llegó en 1999.", "Ella habló brevemente."],
        ...     }
        ... )
        >>> diagnose_alignment(frame).anchor_agreement
        1.0
    """
    agreement, scorable = anchor_agreement(frame, sample, src_col, tgt_col)
    return AlignmentReport(
        length_correlation=round(length_correlation(frame, src_col, tgt_col), 4),
        anchor_agreement=round(agreement, 4),
        scorable_rows=scorable,
        rows=len(frame),
    )

AlignmentReport `dataclass` ¶

AlignmentReport(length_correlation: float, anchor_agreement: float, scorable_rows: int, rows: int)

What the two checks found.

Attributes:

Name	Type	Description
`length_correlation`	`float`	Pearson correlation between source and target sentence lengths in whitespace tokens. Near 0.97 for genuinely parallel text, near 0.0 for unrelated pairs.
`anchor_agreement`	`float`	Fraction of scorable rows whose two sides share at least one name or number. Around 0.39 on the repaired example corpus, 0.014 on the broken one.
`scorable_rows`	`int`	Rows that carried a checkable anchor. If this is small, `anchor_agreement` means little.
`rows`	`int`	Rows examined.

looks_aligned ¶

looks_aligned(min_correlation: float = 0.8, min_agreement: float = 0.25) -> bool

Report whether both checks clear conservative thresholds.

The defaults sit well below what the repaired corpus achieves (0.97 and 0.39) and far above what the broken one managed (0.001 and 0.014). Anything landing between is worth looking at by hand.

Parameters:

Name	Type	Description	Default
`min_correlation`	`float`	Floor for `length_correlation`.	`0.8`
`min_agreement`	`float`	Floor for `anchor_agreement`.	`0.25`

Returns:

Name	Type	Description
`bool`	`bool`	True if both checks pass. A frame with fewer than two usable rows has no measurable length correlation and will not pass whatever its content, so this is a corpus-level check rather than a per-row one.

Source code in src/torchlingo/preprocessing/alignment.py

def looks_aligned(
    self, min_correlation: float = 0.80, min_agreement: float = 0.25
) -> bool:
    """Report whether both checks clear conservative thresholds.

    The defaults sit well below what the repaired corpus achieves (0.97 and
    0.39) and far above what the broken one managed (0.001 and 0.014).
    Anything landing between is worth looking at by hand.

    Args:
        min_correlation (float): Floor for ``length_correlation``.
        min_agreement (float): Floor for ``anchor_agreement``.

    Returns:
        bool: True if both checks pass. A frame with fewer than two usable
            rows has no measurable length correlation and will not pass
            whatever its content, so this is a corpus-level check rather
            than a per-row one.
    """
    return (
        self.length_correlation >= min_correlation
        and self.anchor_agreement >= min_agreement
    )

length_correlation ¶

length_correlation(frame: DataFrame, src_col: str = 'src', tgt_col: str = 'tgt') -> float

Correlate source and target sentence lengths.

Rows where either side is empty are skipped: they carry no length signal and would drag the correlation toward zero for the wrong reason.

Parameters:

Name	Type	Description	Default
`frame`	`DataFrame`	Corpus with source and target columns.	required
`src_col`	`str`	Source column name.	`'src'`
`tgt_col`	`str`	Target column name.	`'tgt'`

Returns:

Name	Type	Description
`float`	`float`	Pearson correlation, or 0.0 if nothing is scorable.

Source code in src/torchlingo/preprocessing/alignment.py

def length_correlation(
    frame: pd.DataFrame, src_col: str = "src", tgt_col: str = "tgt"
) -> float:
    """Correlate source and target sentence lengths.

    Rows where either side is empty are skipped: they carry no length signal
    and would drag the correlation toward zero for the wrong reason.

    Args:
        frame (pd.DataFrame): Corpus with source and target columns.
        src_col (str): Source column name.
        tgt_col (str): Target column name.

    Returns:
        float: Pearson correlation, or 0.0 if nothing is scorable.
    """
    src_len = frame[src_col].astype(str).str.split().str.len()
    tgt_len = frame[tgt_col].astype(str).str.split().str.len()
    usable = (src_len > 0) & (tgt_len > 0)
    if usable.sum() < 2:
        return 0.0
    correlation = src_len[usable].corr(tgt_len[usable])
    return 0.0 if pd.isna(correlation) else float(correlation)

anchor_agreement ¶

anchor_agreement(frame: DataFrame, sample: int = DEFAULT_SAMPLE, src_col: str = 'src', tgt_col: str = 'tgt') -> tuple[float, int]

Measure how often a row's two sides share a name or number.

Scored only on rows whose source side actually contains such a token, since a row with nothing checkable is neither evidence for nor against alignment.

Parameters:

Name	Type	Description	Default
`frame`	`DataFrame`	Corpus with source and target columns.	required
`sample`	`int`	Maximum rows to score.	`DEFAULT_SAMPLE`
`src_col`	`str`	Source column name.	`'src'`
`tgt_col`	`str`	Target column name.	`'tgt'`

Returns:

Type	Description
`tuple[float, int]`	tuple[float, int]: The agreement fraction, and how many rows were scorable. Read the fraction only if the count is meaningful.

Source code in src/torchlingo/preprocessing/alignment.py

def anchor_agreement(
    frame: pd.DataFrame,
    sample: int = DEFAULT_SAMPLE,
    src_col: str = "src",
    tgt_col: str = "tgt",
) -> tuple[float, int]:
    """Measure how often a row's two sides share a name or number.

    Scored only on rows whose source side actually contains such a token, since
    a row with nothing checkable is neither evidence for nor against alignment.

    Args:
        frame (pd.DataFrame): Corpus with source and target columns.
        sample (int): Maximum rows to score.
        src_col (str): Source column name.
        tgt_col (str): Target column name.

    Returns:
        tuple[float, int]: The agreement fraction, and how many rows were
            scorable. Read the fraction only if the count is meaningful.
    """
    hits = total = 0
    sources = frame[src_col].astype(str).head(sample)
    targets = frame[tgt_col].astype(str).head(sample)
    for src, tgt in zip(sources, targets):
        source_anchors = anchors(src)
        if not source_anchors:
            continue
        total += 1
        hits += bool(source_anchors & anchors(tgt))
    return (hits / total if total else 0.0), total

anchors ¶

anchors(text: str) -> set[str]

Extract tokens that should survive translation.

Parameters:

Name	Type	Description	Default
`text`	`str`	A sentence.	required

Returns:

Type	Description
`set[str]`	set[str]: Capitalized words and digit runs found in it.

Example

sorted(anchors("Stephen spoke in 2010.")) ['2010', 'Stephen']

Source code in src/torchlingo/preprocessing/alignment.py

def anchors(text: str) -> set[str]:
    """Extract tokens that should survive translation.

    Args:
        text (str): A sentence.

    Returns:
        set[str]: Capitalized words and digit runs found in it.

    Example:
        >>> sorted(anchors("Stephen spoke in 2010."))
        ['2010', 'Stephen']
    """
    return set(_NAMES.findall(text)) | set(_DIGITS.findall(text))

shuffle_target_side ¶

shuffle_target_side(frame: DataFrame, tgt_col: str = 'tgt') -> DataFrame

Return a copy whose target side no longer matches its source side.

This reconstructs the failure: the same two columns, each individually intact, paired up wrongly. It is how the checks above can be demonstrated on any corpus rather than described in the abstract, and it is what the broken data/example.tsv amounted to.

Parameters:

Name	Type	Description	Default
`frame`	`DataFrame`	An aligned corpus.	required
`tgt_col`	`str`	Target column to rotate.	`'tgt'`

Returns:

Type	Description
`DataFrame`	pd.DataFrame: A copy with the target column shifted by one row, so no row keeps its own translation.

Example

import pandas as pd frame = pd.DataFrame({"src": ["a", "b"], "tgt": ["x", "y"]}) list(shuffle_target_side(frame)["tgt"]) ['y', 'x']

Source code in src/torchlingo/preprocessing/alignment.py

def shuffle_target_side(frame: pd.DataFrame, tgt_col: str = "tgt") -> pd.DataFrame:
    """Return a copy whose target side no longer matches its source side.

    This reconstructs the failure: the same two columns, each individually
    intact, paired up wrongly. It is how the checks above can be demonstrated
    on any corpus rather than described in the abstract, and it is what the
    broken `data/example.tsv` amounted to.

    Args:
        frame (pd.DataFrame): An aligned corpus.
        tgt_col (str): Target column to rotate.

    Returns:
        pd.DataFrame: A copy with the target column shifted by one row, so no
            row keeps its own translation.

    Example:
        >>> import pandas as pd
        >>> frame = pd.DataFrame({"src": ["a", "b"], "tgt": ["x", "y"]})
        >>> list(shuffle_target_side(frame)["tgt"])
        ['y', 'x']
    """
    broken = frame.copy()
    rotated = list(broken[tgt_col])
    broken[tgt_col] = rotated[1:] + rotated[:1]
    return broken

Repairing drift¶

align_one_to_one ¶

align_one_to_one(source: list[str], target: list[str]) -> list[tuple[str, str]]

Align two sentence lists and keep only the confident pairings.

Wraps :func:gale_church_align and returns just the one-to-one beads. Merged and dropped sentences are discarded rather than concatenated: this corpus got into trouble by guessing at alignment, and a teaching corpus is better small and correct than large and uncertain.

Parameters:

Name	Type	Description	Default
`source`	`list[str]`	Source sentences in order.	required
`target`	`list[str]`	Target sentences in order.	required

Returns:

Type	Description
`list[tuple[str, str]]`	list[tuple[str, str]]: Confidently paired sentences.

Example

pairs = align_one_to_one(["A short line."], ["Una linea corta."]) pairs == [("A short line.", "Una linea corta.")] True

Source code in src/torchlingo/preprocessing/alignment.py

def align_one_to_one(source: list[str], target: list[str]) -> list[tuple[str, str]]:
    """Align two sentence lists and keep only the confident pairings.

    Wraps :func:`gale_church_align` and returns just the one-to-one beads.
    Merged and dropped sentences are discarded rather than concatenated: this
    corpus got into trouble by guessing at alignment, and a teaching corpus is
    better small and correct than large and uncertain.

    Args:
        source (list[str]): Source sentences in order.
        target (list[str]): Target sentences in order.

    Returns:
        list[tuple[str, str]]: Confidently paired sentences.

    Example:
        >>> pairs = align_one_to_one(["A short line."], ["Una linea corta."])
        >>> pairs == [("A short line.", "Una linea corta.")]
        True
    """
    return [
        (source[src_idx[0]], target[tgt_idx[0]])
        for src_idx, tgt_idx in gale_church_align(source, target)
        if len(src_idx) == 1 and len(tgt_idx) == 1
    ]

gale_church_align ¶

gale_church_align(source: list[str], target: list[str]) -> list[tuple[list[int], list[int]]]

Align two lists of sentences by length, allowing for slight drift.

Finds the lowest-cost way to walk both sides at once, where each step is a "bead" pairing some sentences on the left with some on the right. Only the patterns in :data:BEAD_COSTS are allowed, which covers the ways segmentation usually differs: a sentence split in two, a sentence dropped, or two sentences merged.

Parameters:

Name	Type	Description	Default
`source`	`list[str]`	Source sentences in order.	required
`target`	`list[str]`	Target sentences in order.	required

Returns:

Type	Description
`list[tuple[list[int], list[int]]]`	list[tuple[list[int], list[int]]]: One entry per bead, holding the source indices and target indices it pairs. A 1-0 bead has an empty target list, and vice versa.

Example

src = ["Hello there.", "How are you?", "Fine."] tgt = ["Hola.", "How are you?", "Bien."] beads = gale_church_align(src, tgt) len(beads) 3 beads[0] == ([0], [0]) True

Source code in src/torchlingo/preprocessing/alignment.py

def gale_church_align(
    source: list[str], target: list[str]
) -> list[tuple[list[int], list[int]]]:
    """Align two lists of sentences by length, allowing for slight drift.

    Finds the lowest-cost way to walk both sides at once, where each step is a
    "bead" pairing some sentences on the left with some on the right. Only the
    patterns in :data:`BEAD_COSTS` are allowed, which covers the ways
    segmentation usually differs: a sentence split in two, a sentence dropped,
    or two sentences merged.

    Args:
        source (list[str]): Source sentences in order.
        target (list[str]): Target sentences in order.

    Returns:
        list[tuple[list[int], list[int]]]: One entry per bead, holding the
            source indices and target indices it pairs. A 1-0 bead has an empty
            target list, and vice versa.

    Example:
        >>> src = ["Hello there.", "How are you?", "Fine."]
        >>> tgt = ["Hola.", "How are you?", "Bien."]
        >>> beads = gale_church_align(src, tgt)
        >>> len(beads)
        3
        >>> beads[0] == ([0], [0])
        True
    """
    n, m = len(source), len(target)
    mean_ratio, variance = _ratio_statistics(source, target)
    src_lengths = [len(s) for s in source]
    tgt_lengths = [len(t) for t in target]

    # cost[i][j] is the cheapest alignment of the first i source sentences
    # against the first j target sentences. back[i][j] records the bead that
    # achieved it, so the path can be walked out at the end.
    infinity = float("inf")
    cost = [[infinity] * (m + 1) for _ in range(n + 1)]
    back: list[list[tuple[int, int] | None]] = [[None] * (m + 1) for _ in range(n + 1)]
    cost[0][0] = 0.0

    for i in range(n + 1):
        for j in range(m + 1):
            if cost[i][j] == infinity:
                continue
            for (take_src, take_tgt), prior in BEAD_COSTS.items():
                next_i, next_j = i + take_src, j + take_tgt
                if next_i > n or next_j > m:
                    continue
                span_src = sum(src_lengths[i:next_i])
                span_tgt = sum(tgt_lengths[j:next_j])
                candidate = (
                    cost[i][j]
                    + prior
                    + _length_cost(span_src, span_tgt, mean_ratio, variance)
                )
                if candidate < cost[next_i][next_j]:
                    cost[next_i][next_j] = candidate
                    back[next_i][next_j] = (take_src, take_tgt)

    beads: list[tuple[list[int], list[int]]] = []
    i, j = n, m
    while i > 0 or j > 0:
        step = back[i][j]
        if step is None:
            # No path reached this cell, which can only happen if the allowed
            # bead patterns cannot span the input. Give up rather than return
            # a partial alignment that looks complete.
            return []
        take_src, take_tgt = step
        beads.append((list(range(i - take_src, i)), list(range(j - take_tgt, j))))
        i, j = i - take_src, j - take_tgt
    beads.reverse()
    return beads

Examples¶

Checking a corpus before you train on it¶

import pandas as pd
from torchlingo.preprocessing import diagnose_alignment

frame = pd.read_csv("data/example.tsv", sep="\t")
report = diagnose_alignment(frame)

print(report)
# AlignmentReport(length_correlation=0.9694, anchor_agreement=0.4102,
#                 scorable_rows=36821, rows=73082)

if not report.looks_aligned():
    raise SystemExit(f"corpus looks misaligned: {report}")

Custom column names¶

report = diagnose_alignment(frame, src_col="english", tgt_col="spanish")

Seeing the checks fail¶

A check you have never watched fail is a check you cannot read. Break a corpus you know is good and run the checks again:

from torchlingo.preprocessing import shuffle_target_side

broken = shuffle_target_side(frame)
print(diagnose_alignment(broken).looks_aligned())   # False

shuffle_target_side rotates the target column by one row. Every column stays individually intact and every pairing breaks, which is what misalignment is.

Adjusting the thresholds¶

The defaults are deliberately conservative: well below what a good corpus achieves, far above what a broken one manages. A noisier corpus or a more distant language pair may legitimately score lower.

report.looks_aligned(min_correlation=0.6, min_agreement=0.15)

Limitations¶

The anchor check needs distinctive anchors. It compares which names and numbers the two sides share, so a token appearing in nearly every row carries no signal. A single speaker's name, repeated throughout their own talks, is shared by every pairing whether that pairing is right or wrong. Scramble such a corpus and agreement stays at 100% while the length correlation collapses.

The length check needs varied lengths. A corpus of uniformly short sentences has little length signal to correlate.

Neither is proof. A corpus can pass both and still be subtly misaligned, for example if it is offset by a whole document rather than scrambled. Look at the numbers, then look at some rows.

Alignment Checks¶

Overview¶

API Reference¶

diagnose_alignment ¶

AlignmentReport dataclass ¶

looks_aligned ¶

length_correlation ¶

anchor_agreement ¶

anchors ¶

shuffle_target_side ¶

Repairing drift¶

align_one_to_one ¶

gale_church_align ¶

Examples¶

Checking a corpus before you train on it¶

Custom column names¶

Seeing the checks fail¶

Adjusting the thresholds¶

Limitations¶

AlignmentReport `dataclass` ¶