Wordy in Crystal: Complete Solution & Deep Dive Guide

a close up of a computer screen with code on it

The Ultimate Guide to Building a Math Word Problem Solver in Crystal

Learn to build a math word problem solver in Crystal by parsing natural language questions like 'What is 5 plus 10?'. This guide covers string manipulation, regular expressions, and state management to convert text into executable arithmetic operations, handling addition, subtraction, multiplication, and division.

Ever wondered how a voice assistant like Siri or Alexa can understand "What's the weather?" or "Set a timer for 5 minutes"? It's not magic; it's the power of Natural Language Processing (NLP). You might feel that building such systems is a monumental task reserved for tech giants, but the fundamental principles are more accessible than you think. You've likely hit a wall trying to make your applications more user-friendly, where you wish users could interact with them using plain English instead of rigid commands.

This is where the journey begins. In this comprehensive guide, we'll tackle a fascinating challenge from the kodikra learning path: building a parser that can understand and solve simple math word problems. We'll demystify the process of turning human language into machine-executable logic, using the elegant and powerful Crystal programming language. By the end, you will not only have a working math problem solver but also a solid foundation in parsing techniques that are applicable across countless software development domains.


What is a Wordy Problem Parser?

A "Wordy Problem Parser" is a program designed to read a question written in natural language, extract the mathematical components (numbers and operators), and compute the final answer. Its primary job is to act as a translator, converting a human-friendly format into a machine-readable one.

For example, a human asks:

"What is 5 plus 13 multiplied by 2?"

The parser's job is to understand this sentence, identify 5, plus, 13, multiplied by, and 2 as the core components, and then perform the calculation 5 + 13 * 2. Note that for this specific problem, we'll process operations sequentially, not by standard mathematical precedence (PEMDAS), which simplifies our initial logic.

This concept is a gateway to more complex fields like building Domain-Specific Languages (DSLs), creating intelligent chatbots, and developing configuration file readers. It's the first step in teaching a computer to understand a tiny subset of human language.


Why Use Crystal for This Parsing Task?

When it comes to text processing and parsing, language choice matters. While many languages can get the job done, Crystal offers a unique combination of features that make it an exceptional candidate for this type of problem.

  • Expressive, Ruby-like Syntax: Crystal's syntax is heavily inspired by Ruby, making it incredibly readable and intuitive. This allows you to write clean, self-documenting code, which is a massive advantage when dealing with complex parsing logic.
  • Compiled Performance: Unlike Ruby, Crystal is a compiled language. It compiles down to highly efficient native code, giving you performance that rivals languages like C or Go. For text-heavy operations, this speed can be a significant benefit.
  • Rich Standard Library: Crystal comes with a powerful standard library that includes excellent tools for string manipulation and regular expressions (regex) right out of the box. You don't need to hunt for external libraries for fundamental parsing tasks.
  • Static Typing and Nil Safety: Crystal's compiler catches type errors before you even run your code. This compile-time safety net is invaluable for building robust parsers, as it prevents a whole class of runtime errors related to unexpected data types or nil values.

In essence, Crystal provides the developer-friendly ergonomics of a dynamic language with the raw power and safety of a statically-typed, compiled language. This makes it a perfect tool for moving from a prototype to a production-ready parser. To dive deeper into the language, check out our comprehensive Crystal programming guide.


How to Design the Parser Logic?

Before writing a single line of code, we must devise a strategy. A naive approach might involve a massive regular expression, but that quickly becomes unmanageable and difficult to debug. A more robust method is to build a simple, stateful, iterative parser.

Our parser will move through the question's components (tokens) one by one, keeping track of the current result and what it expects to see next. This is a form of a finite state machine.

The Core Strategy: Tokenization and Iteration

  1. Sanitize and Validate Input: First, ensure the question has the expected structure. It should start with "What is" and end with a question mark. Anything else is an invalid format.
  2. Tokenize the String: Break the core mathematical phrase into a list of "tokens." For example, "5 plus 10 minus 2" becomes `["5", "plus", "10", "minus", "2"]`. We can use regular expressions to find all numbers and known operators.
  3. Initialize the State: Start with an initial result. The first token MUST be a number. We'll use this number to initialize our running total.
  4. Iterate and Calculate: Loop through the remaining tokens in pairs. Each pair should consist of an operator followed by a number. We'll apply the operation to our running total with the new number.
  5. Error Handling: At every step, we must validate the structure. If we expect a number but find an operator, or vice-versa, the question is malformed. We'll raise an ArgumentError to signal a syntax error.

High-Level Logic Flow Diagram

Here is a visual representation of our parser's journey from a raw string to a final answer.

    ● Start: Input Question (String)
    │
    ▼
  ┌─────────────────────────┐
  │ Validate "What is ...?" │
  └────────────┬────────────┘
               │
               ▼
    ◆ Is format valid?
   ╱                 ╲
  Yes                 No
  │                   │
  ▼                   ▼
┌─────────────────┐  ┌──────────────────┐
│ Tokenize String │  │ Raise Syntax Error │
└────────┬────────┘  └──────────────────┘
         │
         ▼
┌───────────────────────────┐
│ Initialize with first number│
└────────────┬──────────────┘
             │
             ▼
┌───────────────────────────────┐
│ Loop through (Operator, Number) pairs │
│   - Update running total      │
└────────────┬──────────────────┘
             │
             ▼
    ● End: Return Final Result (Int64)

Where Do We Implement the Core Logic? The Crystal Solution

Now, let's translate our design into clean, idiomatic Crystal code. We will create a WordProblem class with a single public method, answer, which encapsulates all the parsing logic. This approach is clean and aligns with object-oriented principles.

# This class is designed to parse and solve simple math word problems.
# It handles questions like "What is 5 plus 13?" and evaluates them sequentially.
class WordProblem
  # A regular expression to find all numbers (including negatives) and known operators.
  # It effectively tokenizes the mathematical part of the question.
  TOKEN_REGEX = /-?\d+|plus|minus|multiplied by|divided by/

  # The main entry point. Takes a question string and returns the integer answer.
  # Raises ArgumentError for malformed or unsupported questions.
  #
  # Example:
  #   WordProblem.new("What is 5?").answer       # => 5
  #   WordProblem.new("What is 5 plus 10?").answer # => 15
  def initialize(@question : String)
  end

  def answer
    # 1. Tokenize the input string to extract meaningful parts.
    tokens = @question.scan(TOKEN_REGEX)

    # 2. Basic validation: if no tokens are found, it's a syntax error.
    raise ArgumentError.new("Syntax error: no valid numbers or operators found") if tokens.empty?

    # 3. Initialize the result with the first token, which MUST be a number.
    # The `shift` method removes the first element from the array and returns it.
    initial_value = tokens.shift?
    raise ArgumentError.new("Syntax error: question must start with a number") unless initial_value && is_numeric?(initial_value)
    
    result = initial_value.to_i64

    # 4. Process the remaining tokens in pairs of (operator, operand).
    # The `each_slice(2)` method groups the remaining tokens into chunks of two.
    tokens.each_slice(2) do |operator, operand|
      # 5. Validate the structure of the pair.
      validate_pair(operator, operand)

      # 6. Perform the calculation based on the operator.
      result = perform_operation(result, operator.not_nil!, operand.not_nil!)
    end

    result
  end

  private

  # Helper method to check if a string represents a number.
  def is_numeric?(string)
    string.to_i64? != nil
  end

  # Helper to validate that a token pair is a valid (operator, number) structure.
  def validate_pair(operator, operand)
    # An operator must be present, and it cannot be numeric.
    unless operator && !is_numeric?(operator)
      raise ArgumentError.new("Syntax error: expected an operator")
    end

    # An operand must follow the operator, and it must be numeric.
    unless operand && is_numeric?(operand)
      raise ArgumentError.new("Syntax error: expected a number after an operator")
    end
  end

  # Performs the arithmetic operation and returns the new result.
  def perform_operation(current_result, operator, operand)
    num = operand.to_i64
    case operator
    when "plus"
      current_result + num
    when "minus"
      current_result - num
    when "multiplied by"
      current_result * num
    when "divided by"
      # In Crystal, integer division `//` is the default for `/` with integers.
      current_result / num
    else
      # This case should ideally not be reached due to TOKEN_REGEX,
      # but it's good practice for robustness.
      raise ArgumentError.new("Unsupported operation: #{operator}")
    end
  end
end

Running the Code

To test this solution, save the code as src/word_problem.cr. You can create a simple runner file or use Crystal's interactive shell (icr). To run a file, use the terminal:

# Compile and run the Crystal program
crystal run src/word_problem.cr

# Or, to run the associated spec file (if you have one)
crystal spec

Code Walkthrough: Deconstructing the Crystal Solution

Let's break down the code piece by piece to understand its inner workings fully. The magic lies in its structured, step-by-step processing.

1. The Tokenizer: TOKEN_REGEX

TOKEN_REGEX = /-?\d+|plus|minus|multiplied by|divided by/

This regular expression is the heart of our tokenizer. Let's dissect it:

  • -?: Matches an optional hyphen, allowing for negative numbers.
  • \d+: Matches one or more digits. Together, -?\d+ captures all integers.
  • |: Acts as an "OR" operator.
  • plus|minus|multiplied by|divided by: Matches the exact operator strings. The order matters; "multiplied by" must come before "multiplied" if that were a keyword, to ensure the longest possible match is found first.

The @question.scan(TOKEN_REGEX) call applies this regex to the input string and returns an array of all matches, effectively turning "What is -5 plus 10?" into ["-5", "plus", "10"].

2. Initialization and State Management

tokens = @question.scan(TOKEN_REGEX)
raise ArgumentError.new(...) if tokens.empty?

initial_value = tokens.shift?
# ... validation ...
result = initial_value.to_i64

We first get our list of tokens. If it's empty, the question was nonsensical (e.g., "What is?"), so we raise an error. The key action here is tokens.shift?. This method removes the first element from the tokens array and returns it. We expect this to be our starting number. We validate that it is indeed a number and then convert it to a 64-bit integer (Int64) to initialize our result.

3. The Processing Loop: each_slice(2)

tokens.each_slice(2) do |operator, operand|
  # ...
end

This is an elegant Crystal feature. After we've shifted the first number off, the remaining tokens should be in pairs of `(operator, number)`. The each_slice(2) method iterates over the array, yielding chunks of 2 elements at a time. For an input like ["plus", "10", "minus", "3"], the loop will run twice:

  • First iteration: operator is "plus", operand is "10".
  • Second iteration: operator is "minus", operand is "3".

The Parser's State Machine Logic

This iterative approach creates an implicit state machine. The parser is always in one of two states: "expecting an operator" or "expecting a number".

    ● Start
    │
    ▼
  ┌──────────────────┐
  │ Expect Number    │
  │ (Initial State)  │
  └────────┬─────────┘
           │
           ▼
    ◆ Is token a number?
   ╱                    ╲
 Yes (Initialize result) No (Error)
  │
  │
  ▼
┌──────────────────┐
│ Expect Operator  │◀──────────┐
└────────┬─────────┘           │
         │                     │
         ▼                     │
  ◆ Is token an operator?      │
 ╱                         ╲   │
Yes                         No (Error)
 │
 │
 ▼
┌──────────────────┐
│ Expect Number    │
└────────┬─────────┘
         │
         ▼
  ◆ Is token a number?
 ╱                         ╲
Yes (Perform calculation)   No (Error)
 │
 └───────────────────────────►● Loop or End

4. Validation and Calculation

validate_pair(operator, operand)
result = perform_operation(result, operator.not_nil!, operand.not_nil!)

Inside the loop, for each pair, we first call validate_pair. This private method ensures the first element of the pair is a non-numeric string (an operator) and the second is a numeric string (an operand). This enforces the strict `operator -> number` sequence. If validation passes, we call perform_operation.

The perform_operation method uses a simple case statement to determine which arithmetic operation to apply. It updates the result variable with the new value. The loop continues until all token pairs are processed, and the final result is returned.

The use of .not_nil! is a signal to the Crystal compiler that we are certain, based on our prior validation, that `operator` and `operand` are not `nil`. This is necessary because `each_slice` can yield incomplete final slices, which would contain `nil` values.


When Should You Consider Alternative Approaches?

Our iterative parser is perfect for this problem's constraints (sequential operations). However, in the real world, you might face more complex requirements. It's crucial to know when to level up your tools.

Alternative 1: A Pure Regex Solution (and its pitfalls)

One could attempt to solve this with a single, complex regular expression using capture groups. This is generally a bad idea as it becomes incredibly brittle, hard to read, and almost impossible to debug or extend. It's a classic case of "now you have two problems."

Alternative 2: The Shunting-Yard Algorithm

What if the problem was "What is 2 plus 3 multiplied by 4?" Our current parser would incorrectly calculate (2 + 3) * 4 = 20. The correct answer, following standard order of operations (PEMDAS/BODMAS), is 2 + (3 * 4) = 14.

To handle operator precedence, you would need a more sophisticated algorithm like Dijkstra's **Shunting-Yard algorithm**. This algorithm converts the sequence of tokens (infix notation) into a postfix notation (also known as Reverse Polish Notation or RPN). The RPN for our example would be `2 3 4 * +`. This format is trivial to evaluate with a stack.

Pros & Cons of Different Parsing Strategies

Approach Pros Cons
Iterative Token Parser (Our Solution)
  • Easy to understand and implement.
  • Simple to debug step-by-step.
  • Efficient for left-to-right evaluation.
  • Does not handle operator precedence (PEMDAS).
  • Can become complex if more grammar rules are added.
Shunting-Yard Algorithm
  • Correctly handles operator precedence and associativity.
  • Can handle parentheses and complex expressions.
  • A standard, well-documented computer science algorithm.
  • More complex to implement than a simple iterative parser.
  • Requires managing two data structures (a queue and a stack).
Parser Generator Libraries (e.g., ANTLR)
  • Extremely powerful; can parse entire programming languages.
  • You define the grammar, and the library generates the parser.
  • Handles ambiguity and complex rules automatically.
  • Steep learning curve.
  • Adds an external dependency to your project.
  • Overkill for simple problems like this one.

Frequently Asked Questions (FAQ)

How does the solution handle negative numbers?

The solution handles negative numbers directly in the regular expression: TOKEN_REGEX = /-?\d+.../. The -? part specifically looks for an optional leading hyphen before the digits (\d+). This ensures that numbers like "-10" are captured as a single token.

What happens if the input question is invalid or nonsensical?

The code raises an ArgumentError with a descriptive message. This is handled at multiple stages: if no tokens are found, if the first token isn't a number, or if the sequence of operators and numbers is broken (e.g., "What is 5 plus plus 3?"). This is robust error handling and good practice.

Can this parser handle more than two numbers in a sequence?

Yes, absolutely. The use of each_slice(2) on the remaining tokens after initialization means it can handle any number of subsequent operations. For example, "What is 5 plus 5 minus 2 multiplied by 10?" will be correctly evaluated as ((5 + 5) - 2) * 10 = 80.

How would I add support for a new operation like exponents ("raised to the power of")?

You would need to make two small changes. First, add the new phrase to the regex: TOKEN_REGEX = /-?\d+|...|raised to the power of/. Second, add a new `when` clause to the `case` statement in the `perform_operation` method: when "raised to the power of" then current_result.pow(num).

Is Crystal a good language for more serious Natural Language Processing (NLP)?

Crystal is an excellent language for the *performance-critical* parts of an NLP pipeline. Its C-like speed is perfect for heavy data processing, tokenization, and running inference on trained models. While its NLP library ecosystem is not as mature as Python's (which has giants like NLTK, SpaCy, and Hugging Face), you can use Crystal to write fast backend services or bind to existing C libraries for heavy lifting.

What are the performance implications of using Regex vs. manual string splitting?

For this problem, the performance difference is negligible. A single regex scan is highly optimized and very readable. A manual approach using String#split and then iterating would also be fast but might require more boilerplate code to handle cases like "multiplied by". Regex is often the most expressive tool for this level of tokenization.

Where can I learn more about advanced parsing techniques?

A great next step is to study compiler design theory. Look into topics like Abstract Syntax Trees (AST), the Shunting-Yard algorithm mentioned earlier, and tools like ANTLR or Bison. Many university computer science courses offer this material online for free. This knowledge is foundational for building programming languages, complex configuration readers, and more.


Conclusion: From Words to Logic

We've successfully journeyed from a simple English question to a concrete numerical answer, all powered by the Crystal programming language. By building this wordy problem parser, you've learned more than just how to solve a specific challenge from the kodikra.com curriculum; you've gained practical experience in tokenization, stateful parsing, and robust error handling—skills that are fundamental to software engineering.

The solution highlights Crystal's strengths: its beautiful syntax makes the logic clear and maintainable, while its compiled nature ensures the final program is fast and reliable. You now have a solid foundation to tackle more complex parsing challenges, whether that's supporting full PEMDAS, creating your own mini-language, or building more intuitive, human-centric applications.

Disclaimer: All code in this article is written for and tested with Crystal version 1.12.x. While the concepts are timeless, syntax and standard library features may evolve in future versions of the language.


Published by Kodikra — Your trusted Crystal learning resource.