Keywords after dot incorrectly highlighted as SQL keywords

Problem

Keywords after dots in STRUCT field access are incorrectly highlighted as SQL keywords instead of identifiers.

Example in tutorial tab 1:

SELECT
  vehicle_snapshot.make,   -- Both parts same color (correct)
  vehicle_snapshot.model   -- 'model' highlighted as keyword (WRONG)

The model field is highlighted in dark blue (keyword color) because MODEL is a BigQuery keyword (used in CREATE MODEL). However, in this context it's a STRUCT field name, not the keyword.

Affected patterns:

• t.model - model = keyword (wrong)
• t.select - select = keyword (wrong)
• t.from - from = keyword (wrong)
• struct.table - table = keyword (wrong)
• Any identifier.keyword_name pattern

Root Cause

The Monarch tokenizer in monarchLanguage.ts is context-free. When tokenizing vehicle_snapshot.model:

1. vehicle_snapshot → identifier ✓
2. . → delimiter ✓
3. model → checks keyword list → finds MODEL → keyword ✗

The tokenizer doesn't know that after a dot, the identifier is always a field/column name, never a SQL keyword.

File: frontend/src/models/managers/MonacoEditorManager/languages/bigquerySQL/monarchLanguage.ts

// Current behavior - matches keywords regardless of context
{
  regex: /[a-zA-Z_][\w@]*/,
  action: {
    cases: {
      '@keywords': 'keyword',  // MODEL matches here
      '@default': 'identifier',
    },
  },
},

Fix

Add a state transition when a dot is encountered. In the afterDot state, identifiers are always treated as identifiers, never keywords.

Change 1: Add dot rule with state transition

Location: Line ~589 in tokenizer.root (after number patterns, before identifier patterns)

// Number patterns MUST come before dot pattern to handle .5 (decimals starting with dot)
{ regex: /\d*\.\d+([eE][-+]?\d+)?/, action: 'number.float' },
{ regex: /\d+([eE][-+]?\d+)?/, action: 'number' },
// Dot operator - enter afterDot state so next identifier is NOT treated as keyword
// This fixes: vehicle_snapshot.model where 'model' should be field, not MODEL keyword
{ regex: /\./, action: { token: 'delimiter.dot', next: '@afterDot' } },

Change 2: Add afterDot state

Location: After the whitespace state in tokenizer (around line 636)

whitespace: [{ regex: /\s+/, action: 'white' }],
// After dot, treat next identifier as field/column (never keyword)
// Example: vehicle_snapshot.model → 'model' is identifier, not MODEL keyword
afterDot: [
  { include: '@whitespace' },
  // Backtick-quoted identifier after dot
  {
    regex: /`/,
    action: { token: 'identifier.quoted', bracket: '@open', next: '@backtickIdentifier' },
  },
  // Regular identifier after dot - always identifier, NEVER keyword
  { regex: /[a-zA-Z_][\w@]*/, action: { token: 'identifier', next: '@pop' } },
  // Anything else - return to root for proper handling
  { regex: /./, action: { token: '@rematch', next: '@pop' } },
],

Verification

Test Cases

| Query | Before | After |
|-------|--------|-------|
| vehicle_snapshot.model | model = keyword (dark blue) | model = identifier (light blue) |
| t.select | select = keyword | select = identifier |
| a.model.select.from.table | Multiple keywords | All identifiers |
| .5 (float literal) | Number | Number (unchanged) |
| SELECT model FROM t | model = keyword | model = keyword (unchanged, expected) |

Edge Cases Handled

1. Floating point numbers (.5, .123e10): Number patterns come before dot pattern
2. Multiple dots (a.b.c.d): Each dot triggers state, pops after identifier
3. Backtick after dot (` project.dataset `): Backtick handler in afterDot state
4. Whitespace after dot (a. b): Whitespace included in afterDot state

Limitations

1. Function names after dot: SAFE.DIVIDE() - DIVIDE treated as identifier instead of function. Acceptable trade-off - functionality unchanged, just cosmetic.

1. Standalone keywords: SELECT model FROM t - bare model still highlighted as keyword (Monarch is context-free). Semantic tokens layer can override when scope analysis resolves it as a column.