API Reference¶

Here you can find all RDAgent’s interfaces.

RD Loop¶

Research¶

class rdagent.core.proposal.Hypothesis(hypothesis: str, reason: str, concise_reason: str, concise_observation: str, concise_justification: str, concise_knowledge: str)¶

TODO: We may have better name for it.

Name Candidates: - Belief

class rdagent.core.proposal.ExperimentFeedback(reason: str, *, code_change_summary: str | None = None, decision: bool, eda_improvement: str | None = None, exception: Exception | None = None)¶

classmethod from_exception(e: Exception) → ExperimentFeedback¶: A convenient method to create Feedback from an exception.

class rdagent.core.proposal.HypothesisFeedback(reason: str, decision: bool, code_change_summary: str = '', *, observations: str | None = None, hypothesis_evaluation: str | None = None, new_hypothesis: str | None = None, eda_improvement: str | None = None, acceptable: bool | None = None, exception: Exception | None = None)¶

class rdagent.core.proposal.Trace(scen: ASpecificScen, knowledge_base: ASpecificKB | None = None)¶

NodeType¶: alias of tuple[Experiment, ExperimentFeedback]

get_sota_hypothesis_and_experiment() → tuple[Hypothesis | None, Experiment | None]¶: Access the last experiment result, sub-task, and the corresponding hypothesis.

is_selection_new_tree(selection: tuple[int, ...] | None = None) → bool¶: Check if the current trace is a new tree. - selection maybe (-1,) when the dag_parent is empty.

get_parent_exps(selection: tuple[int, ...] | None = None) → list[tuple[Experiment, ExperimentFeedback]]¶: Collect all ancestors of the given selection. The return list follows the order of [root->…->parent->current_node].

sync_dag_parent_and_hist(exp_and_fb: NodeType, cur_loop_id: int) → None¶: Adding corresponding parent index to the dag_parent when the hist is going to be changed. Should be called when the hist is changed.

get_children(parent_idx: int | None = None) → list[NodeType]¶: Get all children nodes for a given parent index. If parent_idx is None, returns the root nodes (experiments starting from scratch).

get_sota_experiment(node_id: int | None = None) → Experiment | None¶: Get the SOTA experiment from the trace by traversing ancestors backwards from node_id.

class rdagent.core.proposal.CheckpointSelector¶

In the trace, we may start from any check point (we’ll represent it as a variable from_checkpoint_idx)

abstract get_selection(trace: Trace) → tuple[int, ...] | None¶

checkpoint_idx represents the place where we want to create a new node. the return value should be the idx of target node (the parent of the new generating node). - (-1, ) represents starting from the latest trial in the trace - default value

NOTE: we don’t encourage to use this option; It is confusing when we have multiple traces.

(idx, ) represents starting from the idx-th trial in the trace.
None represents starting from scratch (start a new trace)
More advanced selection strategies in select.py

class rdagent.core.proposal.SOTAexpSelector¶

Select the SOTA experiment from the trace to submit

abstract get_sota_exp_to_submit(trace: Trace) → Experiment | None¶: Select the SOTA experiment from the trace to submit

class rdagent.core.proposal.ExpPlanner(scen: Scenario)¶

An abstract class for planning the experiment. The planner should generate a plan for the experiment based on the trace.

abstract plan(trace: Trace) → ASpecificPlan¶: Generate a plan for the experiment based on the trace. The plan should be a dictionary that contains the plan to each stage.

class rdagent.core.proposal.ExpGen(scen: Scenario)¶

abstract gen(trace: Trace) → Experiment¶

Generate the experiment based on the trace. Planning is part of gen, but since we may support multi-stage planning, we need to pass plan as optional argument.

ExpGen().gen() play a role like

# ExpGen().gen() ==
Hypothesis2Experiment().convert(
    HypothesisGen().gen(trace)
)

async async_gen(trace: Trace, loop: LoopBase) → Experiment¶: generate the experiment and decide whether to stop yield generation and give up control to other routines.

reset() → None¶: Reset the proposal to the initial state. Sometimes the main loop may want to reset the whole process to the initial state. Default implementation does nothing; override in subclasses if needed.

class rdagent.core.proposal.HypothesisGen(scen: Scenario)¶

abstract gen(trace: Trace, plan: ExperimentPlan | None = None) → Hypothesis¶

Motivation of the variable scenario_desc:

Mocking a data-scientist is observing the scenario.

scenario_desc may include:

data observation:
- Original or derivative
Task information:

class rdagent.core.proposal.Hypothesis2Experiment¶

[Abstract description => concrete description] => Code implementation Card

abstract convert(hypothesis: Hypothesis, trace: Trace) → ASpecificExp¶: Connect the idea proposal to implementation

class rdagent.core.proposal.Experiment2Feedback(scen: Scenario)¶

“Generated feedbacks on the hypothesis from Executed Implementations of different tasks & their comparisons with previous performances

abstract generate_feedback(exp: Experiment, trace: Trace, exception: Exception | None = None) → ExperimentFeedback¶: The exp should be executed and the results should be included, as well as the comparison between previous results (done by LLM). For example: mlflow of Qlib will be included.