Skip to content
← Debugging Tools Documentation Generation →

Testing Framework for Dana

(This document is a placeholder. It will propose a testing framework or guidelines for writing and running tests for Dana programs, functions, and agents. This is crucial for ensuring reliability and maintainability.)

Key Considerations for a Dana Testing Framework:

  • Test Structure and Organization:
  • How test files are named and organized (e.g., _test.dna suffix, a dedicated tests/ directory).
  • Defining test cases (e.g., as Dana functions with a specific prefix like test_).
  • Test Runner:
  • A command-line tool to discover and execute tests.
  • Reporting test results (pass, fail, errors, skipped).
  • Options for verbosity, test filtering (by name, tags).
  • Assertions:
  • Built-in assertion functions or a dedicated assert statement in Dana (e.g., assert_equal(actual, expected), assert_true(condition), assert_raises(ErrorType, func_to_call)).
  • Clear error messages on assertion failures.
  • Fixtures and Setup/Teardown:
  • Mechanisms for setting up pre-conditions for tests and cleaning up afterwards.
  • Defining fixtures that can be reused across tests (e.g., a pre-configured SandboxContext or mock resources).
  • Test-level or module-level setup/teardown functions.
  • Mocking and Patching:
  • Ability to mock Resource methods to isolate tests from external dependencies (LLMs, APIs, databases).
  • Mocking Dana functions.
  • Spying on function calls (verifying if a function was called with specific arguments).
  • Testing SandboxContext State:
  • Assertions for checking variable values and types within different scopes of the SandboxContext after a test.
  • Testing IPV-Enabled Functions:
  • Strategies for testing the INFER, PROCESS, and VALIDATE phases independently or together.
  • Mocking CodeContextAnalyzer or LLM calls within IPV.
  • Testing Agent Behaviors:
  • Higher-level testing for agent decision-making and task orchestration.
  • Potentially scenario-based testing where an agent is given an initial state and goal, and its actions/outcomes are verified.
  • Integration with Dana REPL:
  • Ability to quickly run individual tests or test files from the REPL.
  • Code Coverage:
  • (Future) Tools to measure test coverage for Dana code.
  • Parametrized Tests:
  • Running the same test logic with different sets of input data.
  • Tagging and Grouping Tests:
  • Annotating tests with tags (e.g., unit, integration, slow) for selective execution.

Example (Conceptual Dana Test):

# in file: my_module_test.dna

# Import module to test (syntax TBD)
# import my_module

func setup_module():
 # Runs once before tests in this file
 system:test_resource = FileSystemResource(base_path: "/tmp/test_dana_fs")
 log("Module setup complete.")

func teardown_module():
 # Runs once after tests in this file
 # system:test_resource.cleanup() # example method
 log("Module teardown complete.")

func test_my_function_adds_correctly():
 private:result = my_module:add(2, 3)
 assert_equal(result, 5, "Addition did not produce 5")

func test_another_feature_with_mocking():
 # Assume a way to mock a resource method for this test's scope
 # system:mock(system:my_llm_resource, "generate_text", returns: "mocked response")

 private:output = my_module:process_with_llm("some input")
 assert_contains(output, "mocked", "LLM output not used as expected")
 # assert_called_with(system:my_llm_resource.generate_text, "expected prompt")

Self-reflection: A good testing story is essential for any serious programming language or platform. The Dana testing framework should be intuitive for developers, integrate well with Dana's features (like SandboxContext and Resources), and provide robust support for mocking dependencies.