TokenVector Language Support

Official VS Code language support for TokenVector (.tkv, .tv) — an Ahead-Of-Time (AOT) compiled, statically-typed programming language targeting .NET CIL with unboxed scalar types, zero external runtime dependency, and zero-allocation memory performance.

---

⚡ Quick Start (3 Easy Steps)

Get up and running with TokenVector in less than 2 minutes:

1. Install Extension

Search for TokenVector Language Support in the VS Code Extensions Marketplace (Ctrl+Shift+X) and click Install.

2. Write Your First Program (hello.tkv)

Create a new file hello.tkv and type tkv-entry + Tab (or write the code below):

# -*- coding: utf-8 -*-
# hello.tkv - First Native TokenVector Application

def calculate_sum(a: "i32", b: "i32") -> "i32":
    return a + b

def run() -> "str":
    print("=== TokenVector Native Application Initialized ===")
    a = 15
    b = 25
    c = calculate_sum(a, b)
    print("Computed Result a + b = " + str(c))
    return "SUCCESS"

3. Get the Compiler tkvc-tv.exe & Run a Native Binary

1. Clone the official TokenVector repository to obtain the compiler dist\tkvc-tv.exe and the standard libraries (examples/stdlib/):

   git clone https://github.com/nguyenhungtran18/TokenVector.git

2. Compile directly to a standalone native PE executable (.exe):

   # Compile standalone source code - CLI la POSITIONAL:
   #   tkvc-tv.exe <nguon.tkv> <ra.exe> <ten_ham_entry>
   ./tkvc-tv.exe hello.tkv hello.exe run

3. Run standalone binary:

   .\hello.exe

Console Output:

=== TokenVector Native Application Initialized ===
Computed Result a + b = 40
SUCCESS

• Binary Footprint: about 6 KB (hello-world), zero external runtime dependency, true no-GIL multithreading.

---

✨ Features

• 🎨 Rich Syntax Highlighting:

  • Keywords & Control Flow: def, class, return, if, elif, else, for, while, import, from, with, yield, async, await, lambda, raise, try, except, finally, pass, break, continue.
  • Unboxed scalar types: i32, i64, f32, f64, str, plus the aliases int (integer) and bool. Verified by compiling against the real compiler - there are no i8/i16/u*/f16/float/void types.
  • Compiler Identifiers: __name__, __file__, self.
  • Functions, Classes & Records: Full lexical highlighting for definitions, method calls, typed class fields, and inheritance.
  • Operators & Literals: Arithmetic, in-place assignments, comparison, arrow notation (->), hex and floating-point numbers.

• ⚡ Productivity Code Snippets:

  • tkv-entry → Boilerplate application entry point (run() function).
  • tkv-func → Function declaration with unboxed type annotations.
  • tkv-class → Class definition with typed fields and methods.

• 📐 Smart Indentation & Bracket Matching:

  • Automatic 4-space indentation following block headers (def, class, if, elif, else, for, while, try, except).
  • Auto-closing pairs for {}, [], (), "", ''.

---

💻 Code Example: Object-Oriented Class with Typed Fields

# -*- coding: utf-8 -*-
# vector.tkv - 2D Vector Calculation in TokenVector

class Vector2D:
    x: "f64"
    y: "f64"

    def __init__(self, x, y):
        self.x = x
        self.y = y

    def magnitude_squared(self) -> "f64":
        return self.x * self.x + self.y * self.y

def run() -> "str":
    v = Vector2D(3.0, 4.0)
    mag_sq = v.magnitude_squared()
    print("Vector2D magnitude squared: " + str(mag_sq))
    return "SUCCESS"

---

🏛️ Ecosystem & Official Repositories

• ⚡ TokenVector Compiler & Standard Library: https://github.com/nguyenhungtran18/TokenVector
• 🔢 TokenVector.Numerics Mathematical Core Engine: https://github.com/nguyenhungtran18/TokenVector.Numerics
• 🎨 TokenVector VS Code Extension: https://marketplace.visualstudio.com/items?itemName=nguyenhungtran18.tokenvector-syntax

---

📋 Scope & Configuration

• Language ID: tokenvector
• File Extensions: .tkv, .tv
• Scope Name: source.tokenvector
• Compiler Toolchain: tkvc-tv.exe - self-hosted TokenVector compiler; CLI is positional: tkvc-tv.exe <src.tkv> <out.exe> <entry>
• Official Repository: https://github.com/nguyenhungtran18/TokenVector

---

🤝 Contributing & Feedback

Contributions, bug reports, and syntax suggestions are welcome on the TokenVector GitHub Repository.

---

📄 License

MIT License © 2026 TokenVector Project / Nguyen Hung Tran.