Patch: Improving Code Scalability: Refactoring Our Parser into Modular Components

At the core of any compiler or language toolchain, the parser is often one of the most complex and rapidly evolving components. As our project has grown, so has our parser.rs file. To ensure long-term maintainability and to empower our contributors, we have recently completed a significant refactoring of our parser architecture.

The Challenge: Managing Complexity

Previously, a single parser.rs file handled almost every aspect of our language's syntax. While this worked in the early stages, the file eventually grew too large to navigate efficiently. Adding new language features or debugging existing logic became increasingly difficult as different parsing rules were tightly coupled within a single scope.

The Solution: A Modular Architecture

We have decoupled the monolithic parser into a series of logical submodules. This change moves the parser into a dedicated directory structure under front/parser/src/parser/, making the codebase more intuitive and easier to extend.

Here is a breakdown of the new module structure:

• asm.rs: Handles assembly block parsing (parse_asm_block).

• control.rs: Manages control flow structures like if, while, and for loops.

• decl.rs: Centralizes declarations for variables, constants, and keywords like let and var.

• expr.rs: Dedicated to expression parsing, including function calls and parentheses handling.

• functions.rs: Focused on function definitions, parameter parsing, and body extraction.

• io.rs: Streamlines built-in I/O operations such as println, print, and input.

• items.rs: Manages high-level items like imports, protocols, and structs.

• stmt.rs: Orchestrates statements, assignments, and block logic.

• types.rs: A new home for type-parsing logic (formerly in type_system.rs), centralizing how the parser interprets types from tokens.

Key Enhancements & Cleanup

Beyond just moving files, this refactor allowed us to polish the surrounding codebase:

• Unified Entry Point: parse.rs now serves as the main entry point, housing the core parse() function and managing submodule declarations.

• Refined Type System Integration: By merging type_system.rs into types.rs, we’ve created a more cohesive workflow for type resolution during the parsing phase.

• Cleaner Imports: We updated all internal crate imports and resolved unused import warnings in main.rs and runner.rs, resulting in a cleaner build output.

Moving Forward

This refactoring represents a major step forward in our infrastructure. By decoupling the parser’s components, we’ve made it significantly easier for developers to locate specific logic and implement new language features without side effects.

We believe these changes will accelerate our development velocity and provide a more robust foundation for the future of the project.

Link

• Pull request

• GitHub

• Community