{"family":"causal_intervention","level":0,"split":"dev","seed":900000,"instance_hash":"960a86993a361acc1b4870475c7010e812c294ba5786c2d12354287f4c157857","observation":{"variables":["growth","humidity","pressure","turbidity"],"target":"growth","candidates":["humidity","pressure","turbidity"],"intervention_range":{"humidity":[3.5,13.3],"pressure":[6.6,18.2],"growth":[10.6,33.2],"turbidity":[16.2,27.8]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on growth has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"direct_causes","instruction":"find the set of direct causes of growth: the variables that appear with a nonzero coefficient in growth's own structural equation (not causes acting only through other variables, not effects of growth, not merely correlated variables)","report":["causes"],"success_rule":"the COMMIT succeeds only if the committed set equals the set of direct causes exactly; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 4 interventions. Then: COMMIT causes=<comma-separated variable names> (or COMMIT causes=none).","experiment_budget":{"maximum":4,"used":0,"remaining":4},"history":[{"kind":"observational","n":200,"mean":{"growth":21.88,"humidity":8.38,"pressure":12.39,"turbidity":21.94},"cov":[[8.343,3.308,3.979,3.213],[3.308,1.49,1.665,1.348],[3.979,1.665,2.135,1.723],[3.213,1.348,1.723,1.914]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":7,"python_runs":4},"hidden_truth":{"question":{"kind":"direct_causes"},"direct_causes_of_each_variable":{"growth":["pressure"],"humidity":[],"pressure":["humidity"],"turbidity":["pressure"]},"variables_driven_by_the_unobserved_common_cause":["growth","humidity"],"answer":["pressure"]},"gold_commit":"COMMIT causes=pressure","gold_commit_verified":true,"truth_sha256":"e5b7eb40cb962d8268a252485dea3ba6624088297b75cdc75cc4b4bc652e3b9c"}
{"family":"causal_intervention","level":0,"split":"dev","seed":900001,"instance_hash":"facb8c06c2734022b0f0a5707e218451a1e2fa4f813023b0b2c6d651e74a06c6","observation":{"variables":["conductivity","humidity","pressure","titer"],"target":"titer","candidates":["conductivity","humidity","pressure"],"intervention_range":{"pressure":[26.2,37.6],"conductivity":[27.5,40.7],"titer":[31.7,43.1],"humidity":[13.1,22.5]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on titer has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"joint_effect","do":{"conductivity":39.2,"pressure":28.4},"tolerance":1.02,"instruction":"predict the mean of titer under the joint intervention do(conductivity=39.2, pressure=28.4), both variables set at once; the laboratory can set only one variable per batch","report":["value"],"success_rule":"the COMMIT succeeds only if |value - true mean of titer under do(conductivity=39.2, pressure=28.4)| <= 1.02; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 3 interventions. Then: COMMIT value=<number> (the predicted mean of titer).","experiment_budget":{"maximum":3,"used":0,"remaining":3},"history":[{"kind":"observational","n":200,"mean":{"conductivity":34.0,"humidity":17.7,"pressure":31.97,"titer":37.4},"cov":[[2.445,1.369,-1.915,-0.6784],[1.369,1.308,-0.7395,-0.822],[-1.915,-0.7395,1.961,-0.06895],[-0.6784,-0.822,-0.06895,1.82]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":6,"python_runs":4},"hidden_truth":{"question":{"kind":"joint_effect","do":{"conductivity":39.2,"pressure":28.4},"tolerance":1.02},"direct_causes_of_each_variable":{"conductivity":["pressure"],"humidity":["conductivity","pressure"],"pressure":[],"titer":["conductivity","pressure"]},"variables_driven_by_the_unobserved_common_cause":["pressure","titer"],"answer":33.035},"gold_commit":"COMMIT value=33.035","gold_commit_verified":true,"truth_sha256":"620ad8ab9e7037ebdcb87c8affbc4f0b70abea57e8a5f160a0f595cdd273549c"}
{"family":"causal_intervention","level":0,"split":"dev","seed":900002,"instance_hash":"706f0c572763afc39ddd70adc89a38756d79384a81098083f952da7dbd6fb829","observation":{"variables":["conductivity","humidity","stir_rate","yield"],"target":"yield","candidates":["conductivity","humidity","stir_rate"],"intervention_range":{"stir_rate":[29.6,34.4],"conductivity":[17.7,28.5],"yield":[33.2,39.8],"humidity":[25.5,48.5]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on yield has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"joint_effect","do":{"conductivity":19.3,"stir_rate":33.7},"tolerance":0.57,"instruction":"predict the mean of yield under the joint intervention do(conductivity=19.3, stir_rate=33.7), both variables set at once; the laboratory can set only one variable per batch","report":["value"],"success_rule":"the COMMIT succeeds only if |value - true mean of yield under do(conductivity=19.3, stir_rate=33.7)| <= 0.57; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 3 interventions. Then: COMMIT value=<number> (the predicted mean of yield).","experiment_budget":{"maximum":3,"used":0,"remaining":3},"history":[{"kind":"observational","n":200,"mean":{"conductivity":23.25,"humidity":37.23,"stir_rate":32.03,"yield":36.62},"cov":[[1.795,3.556,-0.4304,0.1174],[3.556,7.895,-0.8867,-0.154],[-0.4304,-0.8867,0.4084,-0.004474],[0.1174,-0.154,-0.004474,0.6609]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":6,"python_runs":4},"hidden_truth":{"question":{"kind":"joint_effect","do":{"conductivity":19.3,"stir_rate":33.7},"tolerance":0.57},"direct_causes_of_each_variable":{"conductivity":["stir_rate"],"humidity":["conductivity","stir_rate"],"stir_rate":[],"yield":["conductivity","stir_rate"]},"variables_driven_by_the_unobserved_common_cause":["conductivity","humidity","yield"],"answer":35.2},"gold_commit":"COMMIT value=35.2","gold_commit_verified":true,"truth_sha256":"fedafe59fdadf80b80524bc86fffcf9eae43e8b72028d9a0bc57dee65cc38af7"}
{"family":"causal_intervention","level":0,"split":"dev","seed":900003,"instance_hash":"dfce926062d2e0f83e5b475e5143e327f78df9563c04d3a6023a866a7fbab96b","observation":{"variables":["catalyst","growth","inlet_temp","salt_conc"],"target":"growth","candidates":["catalyst","inlet_temp","salt_conc"],"intervention_range":{"catalyst":[27.8,32.6],"inlet_temp":[3.2,13.0],"salt_conc":[14.7,22.9],"growth":[27.0,39.8]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on growth has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"joint_effect","do":{"inlet_temp":12.5,"salt_conc":15.3},"tolerance":1.3,"instruction":"predict the mean of growth under the joint intervention do(inlet_temp=12.5, salt_conc=15.3), both variables set at once; the laboratory can set only one variable per batch","report":["value"],"success_rule":"the COMMIT succeeds only if |value - true mean of growth under do(inlet_temp=12.5, salt_conc=15.3)| <= 1.3; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 3 interventions. Then: COMMIT value=<number> (the predicted mean of growth).","experiment_budget":{"maximum":3,"used":0,"remaining":3},"history":[{"kind":"observational","n":200,"mean":{"catalyst":30.1,"growth":33.5,"inlet_temp":8.02,"salt_conc":18.72},"cov":[[0.3531,-0.01475,0.07994,0.01758],[-0.01475,2.358,-1.29,-1.311],[0.07994,-1.29,1.362,0.8467],[0.01758,-1.311,0.8467,0.9715]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":6,"python_runs":4},"hidden_truth":{"question":{"kind":"joint_effect","do":{"inlet_temp":12.5,"salt_conc":15.3},"tolerance":1.3},"direct_causes_of_each_variable":{"catalyst":[],"growth":["inlet_temp","salt_conc"],"inlet_temp":[],"salt_conc":["inlet_temp"]},"variables_driven_by_the_unobserved_common_cause":["growth","inlet_temp"],"answer":32.023},"gold_commit":"COMMIT value=32.023","gold_commit_verified":true,"truth_sha256":"0522cd6b80ada8c6d4f406b3d95741859661cce5fa154714724e322d00ee441d"}
{"family":"causal_intervention","level":0,"split":"dev","seed":900004,"instance_hash":"d9dcd0b8807169de6ad8178fe24dfb41b0a2799dd064214ba0a6a63111c32ff2","observation":{"variables":["catalyst","growth","pressure","turbidity"],"target":"growth","candidates":["catalyst","pressure","turbidity"],"intervention_range":{"catalyst":[28.9,32.9],"turbidity":[11.4,22.8],"pressure":[9.1,18.3],"growth":[4.4,19.2]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on growth has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"direct_causes","instruction":"find the set of direct causes of growth: the variables that appear with a nonzero coefficient in growth's own structural equation (not causes acting only through other variables, not effects of growth, not merely correlated variables)","report":["causes"],"success_rule":"the COMMIT succeeds only if the committed set equals the set of direct causes exactly; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 4 interventions. Then: COMMIT causes=<comma-separated variable names> (or COMMIT causes=none).","experiment_budget":{"maximum":4,"used":0,"remaining":4},"history":[{"kind":"observational","n":200,"mean":{"catalyst":30.92,"growth":11.84,"pressure":13.68,"turbidity":17.15},"cov":[[0.2646,0.09179,0.172,-0.2084],[0.09179,3.133,-1.391,2.005],[0.172,-1.391,1.214,-1.273],[-0.2084,2.005,-1.273,2.074]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":7,"python_runs":4},"hidden_truth":{"question":{"kind":"direct_causes"},"direct_causes_of_each_variable":{"catalyst":[],"growth":["catalyst","pressure"],"pressure":["turbidity"],"turbidity":["catalyst"]},"variables_driven_by_the_unobserved_common_cause":["growth","turbidity"],"answer":["catalyst","pressure"]},"gold_commit":"COMMIT causes=catalyst,pressure","gold_commit_verified":true,"truth_sha256":"de9dca8f2b8dd195935a7d2545b38eb236a0053cce030a751e4eb4058d96617b"}
{"family":"causal_intervention","level":0,"split":"dev","seed":900005,"instance_hash":"6850ebaee16c5406f989ecf3a6e1c5dcce143b3c1a3bd9fd695e8387ba9b6844","observation":{"variables":["catalyst","growth","stir_rate","turbidity"],"target":"growth","candidates":["catalyst","stir_rate","turbidity"],"intervention_range":{"turbidity":[4.4,9.2],"catalyst":[35.2,44.6],"stir_rate":[27.4,35.0],"growth":[13.0,21.4]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on growth has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"direct_causes","instruction":"find the set of direct causes of growth: the variables that appear with a nonzero coefficient in growth's own structural equation (not causes acting only through other variables, not effects of growth, not merely correlated variables)","report":["causes"],"success_rule":"the COMMIT succeeds only if the committed set equals the set of direct causes exactly; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 4 interventions. Then: COMMIT causes=<comma-separated variable names> (or COMMIT causes=none).","experiment_budget":{"maximum":4,"used":0,"remaining":4},"history":[{"kind":"observational","n":200,"mean":{"catalyst":39.76,"growth":17.25,"stir_rate":31.12,"turbidity":6.86},"cov":[[1.151,-0.2145,0.5642,0.1318],[-0.2145,1.163,0.4485,-0.2017],[0.5642,0.4485,0.9253,-0.2464],[0.1318,-0.2017,-0.2464,0.3019]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":7,"python_runs":4},"hidden_truth":{"question":{"kind":"direct_causes"},"direct_causes_of_each_variable":{"catalyst":["turbidity"],"growth":["stir_rate"],"stir_rate":["catalyst","turbidity"],"turbidity":[]},"variables_driven_by_the_unobserved_common_cause":["catalyst","growth"],"answer":["stir_rate"]},"gold_commit":"COMMIT causes=stir_rate","gold_commit_verified":true,"truth_sha256":"99a80acc03f9ffc763268fbd13bc055b40317630e95b88f74648bb7fb4e9d48e"}
{"family":"causal_intervention","level":0,"split":"dev","seed":900006,"instance_hash":"7ed290fcbb6094e8c736ba37382b99db63089e35522961923548699273367e86","observation":{"variables":["pigment","salt_conc","stir_rate","yield"],"target":"yield","candidates":["pigment","salt_conc","stir_rate"],"intervention_range":{"pigment":[12.6,21.6],"salt_conc":[11.3,20.9],"yield":[9.9,18.9],"stir_rate":[21.6,43.6]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on yield has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"direct_causes","instruction":"find the set of direct causes of yield: the variables that appear with a nonzero coefficient in yield's own structural equation (not causes acting only through other variables, not effects of yield, not merely correlated variables)","report":["causes"],"success_rule":"the COMMIT succeeds only if the committed set equals the set of direct causes exactly; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 4 interventions. Then: COMMIT causes=<comma-separated variable names> (or COMMIT causes=none).","experiment_budget":{"maximum":4,"used":0,"remaining":4},"history":[{"kind":"observational","n":200,"mean":{"pigment":17.01,"salt_conc":16.0,"stir_rate":32.72,"yield":14.42},"cov":[[1.265,1.193,-2.687,0.1972],[1.193,1.401,-2.662,-0.02717],[-2.687,-2.662,8.047,-1.775],[0.1972,-0.02717,-1.775,1.371]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":7,"python_runs":4},"hidden_truth":{"question":{"kind":"direct_causes"},"direct_causes_of_each_variable":{"pigment":[],"salt_conc":["pigment"],"stir_rate":["salt_conc","yield"],"yield":["salt_conc"]},"variables_driven_by_the_unobserved_common_cause":["pigment","stir_rate","yield"],"answer":["salt_conc"]},"gold_commit":"COMMIT causes=salt_conc","gold_commit_verified":true,"truth_sha256":"63c3057378f6363ad2fcf34bf657c1cb18abe6235f1ca1c42fb5b7005c34aa11"}
{"family":"causal_intervention","level":0,"split":"dev","seed":900007,"instance_hash":"9cf2d61a7871ad9481095daf235e91bb24fcf95f6111f75a25c1a76282ed5ccf","observation":{"variables":["catalyst","output","pressure","salt_conc"],"target":"output","candidates":["catalyst","pressure","salt_conc"],"intervention_range":{"pressure":[6.8,18.2],"catalyst":[18.7,32.5],"output":[16.7,26.1],"salt_conc":[24.5,52.1]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on output has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"joint_effect","do":{"catalyst":31.7,"pressure":17.4},"tolerance":1.26,"instruction":"predict the mean of output under the joint intervention do(catalyst=31.7, pressure=17.4), both variables set at once; the laboratory can set only one variable per batch","report":["value"],"success_rule":"the COMMIT succeeds only if |value - true mean of output under do(catalyst=31.7, pressure=17.4)| <= 1.26; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 3 interventions. Then: COMMIT value=<number> (the predicted mean of output).","experiment_budget":{"maximum":3,"used":0,"remaining":3},"history":[{"kind":"observational","n":200,"mean":{"catalyst":25.62,"output":21.48,"pressure":12.5,"salt_conc":38.43},"cov":[[3.11,-1.474,-2.425,-5.448],[-1.474,1.335,1.128,3.317],[-2.425,1.128,2.037,4.49],[-5.448,3.317,4.49,11.29]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":6,"python_runs":4},"hidden_truth":{"question":{"kind":"joint_effect","do":{"catalyst":31.7,"pressure":17.4},"tolerance":1.26},"direct_causes_of_each_variable":{"catalyst":["pressure"],"output":["catalyst","pressure"],"pressure":[],"salt_conc":["output","pressure"]},"variables_driven_by_the_unobserved_common_cause":["output","pressure","salt_conc"],"answer":12.279},"gold_commit":"COMMIT value=12.279","gold_commit_verified":true,"truth_sha256":"35c6c3ed22c426674b78dc56fed30af6f510981a998874d18d85a0e87a0ace32"}
{"family":"causal_intervention","level":1,"split":"dev","seed":900000,"instance_hash":"a95eae9ae9d63820ba87a650e93cd535b689b389ace776a151e19283e4fa3d37","observation":{"variables":["inlet_temp","light_dose","output","redox","turbidity","viscosity"],"target":"output","candidates":["inlet_temp","light_dose","redox","turbidity","viscosity"],"intervention_range":{"light_dose":[6.8,18.0],"turbidity":[0.2,13.8],"output":[33.5,46.5],"inlet_temp":[-0.0,21.2],"redox":[12.0,29.2],"viscosity":[1.6,23.4]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on output has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"joint_effect","do":{"light_dose":15.8,"turbidity":12.9},"tolerance":1.04,"instruction":"predict the mean of output under the joint intervention do(light_dose=15.8, turbidity=12.9), both variables set at once; the laboratory can set only one variable per batch","report":["value"],"success_rule":"the COMMIT succeeds only if |value - true mean of output under do(light_dose=15.8, turbidity=12.9)| <= 1.04; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 2 interventions. Then: COMMIT value=<number> (the predicted mean of output).","experiment_budget":{"maximum":2,"used":0,"remaining":2},"history":[{"kind":"observational","n":200,"mean":{"inlet_temp":10.53,"light_dose":12.41,"output":39.88,"redox":20.64,"turbidity":7.06,"viscosity":12.51},"cov":[[6.292,2.715,3.348,-4.775,2.841,5.727],[2.715,1.855,1.02,-2.109,2.104,3.332],[3.348,1.02,2.499,-2.527,0.7104,2.536],[-4.775,-2.109,-2.527,4.241,-2.189,-4.758],[2.841,2.104,0.7104,-2.189,2.766,3.649],[5.727,3.332,2.536,-4.758,3.649,6.91]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":6,"python_runs":4},"hidden_truth":{"question":{"kind":"joint_effect","do":{"light_dose":15.8,"turbidity":12.9},"tolerance":1.04},"direct_causes_of_each_variable":{"inlet_temp":["output","turbidity"],"light_dose":[],"output":["light_dose","turbidity"],"redox":["inlet_temp"],"turbidity":["light_dose"],"viscosity":["light_dose","redox"]},"variables_driven_by_the_unobserved_common_cause":["light_dose","output"],"answer":36.937},"gold_commit":"COMMIT value=36.937","gold_commit_verified":true,"truth_sha256":"da39b426ae96dd328862ee2d51e24974c58b8780131e15622ab6cf1791a007c3"}
{"family":"causal_intervention","level":1,"split":"dev","seed":900001,"instance_hash":"ca7d6a640e81fdf94b1beb0f9843f7e1b5372f22dd2ada2ce8e0bb6cec90249e","observation":{"variables":["foam_height","ph_level","redox","viscosity","yield"],"target":"yield","candidates":["foam_height","ph_level","redox","viscosity"],"intervention_range":{"foam_height":[19.5,30.3],"redox":[2.6,16.2],"viscosity":[10.5,23.7],"yield":[10.7,23.3],"ph_level":[21.8,31.4]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on yield has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"joint_effect","do":{"foam_height":20.2,"redox":13.4},"tolerance":0.86,"instruction":"predict the mean of yield under the joint intervention do(foam_height=20.2, redox=13.4), both variables set at once; the laboratory can set only one variable per batch","report":["value"],"success_rule":"the COMMIT succeeds only if |value - true mean of yield under do(foam_height=20.2, redox=13.4)| <= 0.86; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 2 interventions. Then: COMMIT value=<number> (the predicted mean of yield).","experiment_budget":{"maximum":2,"used":0,"remaining":2},"history":[{"kind":"observational","n":200,"mean":{"foam_height":24.88,"ph_level":26.62,"redox":9.51,"viscosity":16.98,"yield":16.92},"cov":[[1.848,0.7131,-2.156,1.785,1.068],[0.7131,1.442,-1.077,0.9384,1.526],[-2.156,-1.077,2.861,-2.379,-1.694],[1.785,0.9384,-2.379,2.574,1.478],[1.068,1.526,-1.694,1.478,2.455]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":6,"python_runs":4},"hidden_truth":{"question":{"kind":"joint_effect","do":{"foam_height":20.2,"redox":13.4},"tolerance":0.86},"direct_causes_of_each_variable":{"foam_height":[],"ph_level":["yield"],"redox":["foam_height"],"viscosity":["redox"],"yield":["foam_height","redox"]},"variables_driven_by_the_unobserved_common_cause":["foam_height","yield"],"answer":18.963},"gold_commit":"COMMIT value=18.963","gold_commit_verified":true,"truth_sha256":"2ae8d0ce71acc32c9b9b656c3273b3aef3943330f863ff91890329cd24a10630"}
{"family":"causal_intervention","level":1,"split":"dev","seed":900002,"instance_hash":"e02916a33b5ac1558892d923c0e817ce6f81dd9a5555f08363491d0a8a16adde","observation":{"variables":["output","ph_level","pressure","salt_conc","viscosity"],"target":"output","candidates":["ph_level","pressure","salt_conc","viscosity"],"intervention_range":{"pressure":[16.9,23.3],"ph_level":[16.8,29.8],"output":[12.1,19.9],"viscosity":[4.4,15.4],"salt_conc":[7.5,14.7]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on output has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"joint_effect","do":{"ph_level":27.7,"pressure":22.5},"tolerance":0.58,"instruction":"predict the mean of output under the joint intervention do(ph_level=27.7, pressure=22.5), both variables set at once; the laboratory can set only one variable per batch","report":["value"],"success_rule":"the COMMIT succeeds only if |value - true mean of output under do(ph_level=27.7, pressure=22.5)| <= 0.58; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 2 interventions. Then: COMMIT value=<number> (the predicted mean of output).","experiment_budget":{"maximum":2,"used":0,"remaining":2},"history":[{"kind":"observational","n":200,"mean":{"output":16.07,"ph_level":23.16,"pressure":20.08,"salt_conc":11.14,"viscosity":9.87},"cov":[[1.098,-0.1206,-0.1332,0.03244,1.101],[-0.1206,2.953,0.7882,-0.05634,-0.6234],[-0.1332,0.7882,0.6343,-0.08491,-0.5829],[0.03244,-0.05634,-0.08491,0.7778,0.1971],[1.101,-0.6234,-0.5829,0.1971,2.222]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":6,"python_runs":4},"hidden_truth":{"question":{"kind":"joint_effect","do":{"ph_level":27.7,"pressure":22.5},"tolerance":0.58},"direct_causes_of_each_variable":{"output":["ph_level","pressure"],"ph_level":["pressure"],"pressure":[],"salt_conc":[],"viscosity":["output","pressure"]},"variables_driven_by_the_unobserved_common_cause":["output","ph_level"],"answer":17.028},"gold_commit":"COMMIT value=17.028","gold_commit_verified":true,"truth_sha256":"185ae4bddecb2693f2efaa47be50e513322c0e0d134c77160887060cd35e2ac8"}
{"family":"causal_intervention","level":1,"split":"dev","seed":900003,"instance_hash":"528fb8445a48bfa188ec1774fd6c6f384ac827c44dcf474de7901532e40ccabe","observation":{"variables":["conductivity","feed_flow","light_dose","ph_level","salt_conc","yield"],"target":"yield","candidates":["conductivity","feed_flow","light_dose","ph_level","salt_conc"],"intervention_range":{"conductivity":[16.8,23.0],"feed_flow":[30.4,39.8],"light_dose":[12.2,26.8],"yield":[16.8,43.6],"ph_level":[12.5,33.1],"salt_conc":[11.1,22.1]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on yield has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"joint_effect","do":{"conductivity":18.1,"light_dose":24.8},"tolerance":1.16,"instruction":"predict the mean of yield under the joint intervention do(conductivity=18.1, light_dose=24.8), both variables set at once; the laboratory can set only one variable per batch","report":["value"],"success_rule":"the COMMIT succeeds only if |value - true mean of yield under do(conductivity=18.1, light_dose=24.8)| <= 1.16; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 2 interventions. Then: COMMIT value=<number> (the predicted mean of yield).","experiment_budget":{"maximum":2,"used":0,"remaining":2},"history":[{"kind":"observational","n":200,"mean":{"conductivity":19.9,"feed_flow":35.12,"light_dose":19.6,"ph_level":22.68,"salt_conc":16.59,"yield":30.2},"cov":[[0.6259,0.6953,-0.8235,-0.4074,0.6763,-2.52],[0.6953,1.349,-1.027,-0.333,1.284,-3.534],[-0.8235,-1.027,3.379,-2.57,-0.9114,4.071],[-0.4074,-0.333,-2.57,5.995,-0.3885,1.641],[0.6763,1.284,-0.9114,-0.3885,1.751,-3.295],[-2.52,-3.534,4.071,1.641,-3.295,12.24]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":6,"python_runs":4},"hidden_truth":{"question":{"kind":"joint_effect","do":{"conductivity":18.1,"light_dose":24.8},"tolerance":1.16},"direct_causes_of_each_variable":{"conductivity":[],"feed_flow":["conductivity"],"light_dose":["conductivity"],"ph_level":["feed_flow","light_dose","yield"],"salt_conc":["feed_flow"],"yield":["conductivity","feed_flow","light_dose"]},"variables_driven_by_the_unobserved_common_cause":["light_dose","ph_level","yield"],"answer":40.791},"gold_commit":"COMMIT value=40.791","gold_commit_verified":true,"truth_sha256":"a7e01117190f4976bd4657b43902d959a2f364732f9ebe6b70ba2d0ab21c1245"}
{"family":"causal_intervention","level":1,"split":"dev","seed":900004,"instance_hash":"7e54477a60769ed44f095d995534a0ba1d0ec7dc1e4ed85f0306845daf943a9a","observation":{"variables":["foam_height","humidity","oxygen","pressure","titer"],"target":"titer","candidates":["foam_height","humidity","oxygen","pressure"],"intervention_range":{"humidity":[26.6,31.8],"oxygen":[34.4,41.4],"pressure":[14.1,28.3],"foam_height":[23.4,37.8],"titer":[3.1,15.3]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on titer has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"joint_effect","do":{"foam_height":24.4,"pressure":16.0},"tolerance":1.85,"instruction":"predict the mean of titer under the joint intervention do(foam_height=24.4, pressure=16.0), both variables set at once; the laboratory can set only one variable per batch","report":["value"],"success_rule":"the COMMIT succeeds only if |value - true mean of titer under do(foam_height=24.4, pressure=16.0)| <= 1.85; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 2 interventions. Then: COMMIT value=<number> (the predicted mean of titer).","experiment_budget":{"maximum":2,"used":0,"remaining":2},"history":[{"kind":"observational","n":200,"mean":{"foam_height":30.67,"humidity":29.16,"oxygen":37.97,"pressure":21.22,"titer":9.04},"cov":[[3.213,0.04412,0.54,-2.497,-1.345],[0.04412,0.3972,-0.03025,-0.4227,0.4695],[0.54,-0.03025,0.668,0.1127,0.1275],[-2.497,-0.4227,0.1127,3.157,0.8083],[-1.345,0.4695,0.1275,0.8083,2.598]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":6,"python_runs":4},"hidden_truth":{"question":{"kind":"joint_effect","do":{"foam_height":24.4,"pressure":16.0},"tolerance":1.85},"direct_causes_of_each_variable":{"foam_height":["humidity","oxygen","pressure"],"humidity":[],"oxygen":[],"pressure":["humidity"],"titer":["foam_height","oxygen","pressure"]},"variables_driven_by_the_unobserved_common_cause":["pressure","titer"],"answer":21.3},"gold_commit":"COMMIT value=21.3","gold_commit_verified":true,"truth_sha256":"4bf2b6d8cdbbfb5da3c724240305d80306310a17d9552668fe6d1ced4f36bd7f"}
{"family":"causal_intervention","level":1,"split":"dev","seed":900005,"instance_hash":"91f56d4ce4a4d8b49c1b76c99415a077eba0ec783bc07aa23ad9e43b68659b15","observation":{"variables":["conductivity","foam_height","light_dose","output","pressure"],"target":"output","candidates":["conductivity","foam_height","light_dose","pressure"],"intervention_range":{"light_dose":[20.7,25.9],"conductivity":[9.7,20.1],"pressure":[24.9,34.7],"foam_height":[19.1,31.1],"output":[19.2,40.8]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on output has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"direct_causes","instruction":"find the set of direct causes of output: the variables that appear with a nonzero coefficient in output's own structural equation (not causes acting only through other variables, not effects of output, not merely correlated variables)","report":["causes"],"success_rule":"the COMMIT succeeds only if the committed set equals the set of direct causes exactly; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 4 interventions. Then: COMMIT causes=<comma-separated variable names> (or COMMIT causes=none).","experiment_budget":{"maximum":4,"used":0,"remaining":4},"history":[{"kind":"observational","n":200,"mean":{"conductivity":14.79,"foam_height":25.33,"light_dose":23.2,"output":30.32,"pressure":29.76},"cov":[[1.53,0.08465,-0.1254,-2.32,1.014],[0.08465,2.027,-0.3239,0.8081,0.8181],[-0.1254,-0.3239,0.393,-0.4647,-0.08319],[-2.32,0.8081,-0.4647,6.584,-1.94],[1.014,0.8181,-0.08319,-1.94,1.368]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":7,"python_runs":4},"hidden_truth":{"question":{"kind":"direct_causes"},"direct_causes_of_each_variable":{"conductivity":[],"foam_height":["conductivity","light_dose","pressure"],"light_dose":[],"output":["foam_height","light_dose","pressure"],"pressure":["conductivity"]},"variables_driven_by_the_unobserved_common_cause":["conductivity","output"],"answer":["foam_height","light_dose","pressure"]},"gold_commit":"COMMIT causes=foam_height,light_dose,pressure","gold_commit_verified":true,"truth_sha256":"3d5f4b2e58088d056f4bc2674e532ca92f92286afb30e73d777b7b9440112e03"}
{"family":"causal_intervention","level":1,"split":"dev","seed":900006,"instance_hash":"e520775b6d3b88cfdb996bd4f0ce2e9f8492390808e67cd8c2b691882939fd55","observation":{"variables":["feed_flow","growth","inlet_temp","light_dose","pressure","viscosity"],"target":"growth","candidates":["feed_flow","inlet_temp","light_dose","pressure","viscosity"],"intervention_range":{"light_dose":[8.2,21.0],"viscosity":[20.5,32.5],"pressure":[18.1,32.5],"inlet_temp":[7.8,23.4],"growth":[12.3,29.9],"feed_flow":[-1.1,16.1]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on growth has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"joint_effect","do":{"inlet_temp":20.9,"light_dose":9.5},"tolerance":1.09,"instruction":"predict the mean of growth under the joint intervention do(inlet_temp=20.9, light_dose=9.5), both variables set at once; the laboratory can set only one variable per batch","report":["value"],"success_rule":"the COMMIT succeeds only if |value - true mean of growth under do(inlet_temp=20.9, light_dose=9.5)| <= 1.09; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 2 interventions. Then: COMMIT value=<number> (the predicted mean of growth).","experiment_budget":{"maximum":2,"used":0,"remaining":2},"history":[{"kind":"observational","n":200,"mean":{"feed_flow":7.36,"growth":20.95,"inlet_temp":15.67,"light_dose":14.64,"pressure":25.17,"viscosity":26.52},"cov":[[4.603,0.6278,-3.955,1.355,0.8407,1.334],[0.6278,5.574,-0.6232,-2.559,3.647,-2.641],[-3.955,-0.6232,4.089,-1.399,-0.8427,-1.349],[1.355,-2.559,-1.399,2.524,-2.128,2.175],[0.8407,3.647,-0.8427,-2.128,3.425,-1.897],[1.334,-2.641,-1.349,2.175,-1.897,2.238]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":6,"python_runs":4},"hidden_truth":{"question":{"kind":"joint_effect","do":{"inlet_temp":20.9,"light_dose":9.5},"tolerance":1.09},"direct_causes_of_each_variable":{"feed_flow":["inlet_temp"],"growth":["inlet_temp","light_dose","viscosity"],"inlet_temp":["light_dose","pressure","viscosity"],"light_dose":[],"pressure":[],"viscosity":["light_dose"]},"variables_driven_by_the_unobserved_common_cause":["growth","light_dose","pressure"],"answer":27.844},"gold_commit":"COMMIT value=27.844","gold_commit_verified":true,"truth_sha256":"f8559f40fba08ad51677c8d34515fc889745b2f93a2fbb348b22d0f895a3f85e"}
{"family":"causal_intervention","level":1,"split":"dev","seed":900007,"instance_hash":"27a40bd58208310ac42489ea024c61c95114cd22ea76a245de9e469b119258c5","observation":{"variables":["catalyst","light_dose","purity","salt_conc","turbidity"],"target":"purity","candidates":["catalyst","light_dose","salt_conc","turbidity"],"intervention_range":{"light_dose":[30.2,37.0],"turbidity":[0.8,14.8],"purity":[6.0,23.8],"catalyst":[20.0,49.2],"salt_conc":[24.7,46.5]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on purity has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"joint_effect","do":{"light_dose":31.6,"turbidity":12.2},"tolerance":0.67,"instruction":"predict the mean of purity under the joint intervention do(light_dose=31.6, turbidity=12.2), both variables set at once; the laboratory can set only one variable per batch","report":["value"],"success_rule":"the COMMIT succeeds only if |value - true mean of purity under do(light_dose=31.6, turbidity=12.2)| <= 0.67; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 2 interventions. Then: COMMIT value=<number> (the predicted mean of purity).","experiment_budget":{"maximum":2,"used":0,"remaining":2},"history":[{"kind":"observational","n":200,"mean":{"catalyst":34.92,"light_dose":33.52,"purity":15.04,"salt_conc":35.7,"turbidity":7.87},"cov":[[10.98,-2.474,6.593,6.465,3.074],[-2.474,0.6859,-1.507,-1.349,-0.7011],[6.593,-1.507,4.341,3.995,2.136],[6.465,-1.349,3.995,6.369,3.661],[3.074,-0.7011,2.136,3.661,2.808]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":6,"python_runs":4},"hidden_truth":{"question":{"kind":"joint_effect","do":{"light_dose":31.6,"turbidity":12.2},"tolerance":0.67},"direct_causes_of_each_variable":{"catalyst":["light_dose","purity"],"light_dose":[],"purity":["light_dose","turbidity"],"salt_conc":["catalyst","light_dose"],"turbidity":["light_dose"]},"variables_driven_by_the_unobserved_common_cause":["purity","salt_conc","turbidity"],"answer":21.644},"gold_commit":"COMMIT value=21.644","gold_commit_verified":true,"truth_sha256":"1c1a4acaf417bd732c5de184c3875c5da4475c85b27eb45fdbf07ade7e54989a"}
{"family":"causal_intervention","level":2,"split":"dev","seed":900000,"instance_hash":"583b7d78a311a0f3206a396c321761065bd08e37a484d03ddde62f8676fb9b96","observation":{"variables":["catalyst","foam_height","light_dose","pressure","stir_rate","yield"],"target":"yield","candidates":["catalyst","foam_height","light_dose","pressure","stir_rate"],"intervention_range":{"stir_rate":[8.0,15.6],"light_dose":[15.7,32.1],"pressure":[14.0,21.0],"yield":[7.0,31.2],"foam_height":[-0.8,17.2],"catalyst":[8.2,26.6]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on yield has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"joint_effect","do":{"light_dose":30.6,"stir_rate":14.3},"tolerance":1.1,"instruction":"predict the mean of yield under the joint intervention do(light_dose=30.6, stir_rate=14.3), both variables set at once; the laboratory can set only one variable per batch","report":["value"],"success_rule":"the COMMIT succeeds only if |value - true mean of yield under do(light_dose=30.6, stir_rate=14.3)| <= 1.1; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 2 interventions. Then: COMMIT value=<number> (the predicted mean of yield).","experiment_budget":{"maximum":2,"used":0,"remaining":2},"history":[{"kind":"observational","n":200,"mean":{"catalyst":17.55,"foam_height":8.08,"light_dose":24.15,"pressure":17.52,"stir_rate":11.95,"yield":18.87},"cov":[[7.646,-6.639,1.075,-1.333,1.294,-5.933],[-6.639,6.358,-1.138,1.285,-1.259,5.748],[1.075,-1.138,5.241,0.2231,1.352,-4.798],[-1.333,1.285,0.2231,0.9729,0.1052,0.6133],[1.294,-1.259,1.352,0.1052,1.208,-2.858],[-5.933,5.748,-4.798,0.6133,-2.858,10.12]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":6,"python_runs":4},"hidden_truth":{"question":{"kind":"joint_effect","do":{"light_dose":30.6,"stir_rate":14.3},"tolerance":1.1},"direct_causes_of_each_variable":{"catalyst":["foam_height"],"foam_height":["light_dose","pressure","yield"],"light_dose":["stir_rate"],"pressure":[],"stir_rate":[],"yield":["light_dose","pressure","stir_rate"]},"variables_driven_by_the_unobserved_common_cause":["foam_height","light_dose","yield"],"answer":8.019},"gold_commit":"COMMIT value=8.019","gold_commit_verified":true,"truth_sha256":"c148761a70332efbe39b24517599eb16152ac29191002cdcbd5870c7de230151"}
{"family":"causal_intervention","level":2,"split":"dev","seed":900001,"instance_hash":"a76c298514d3a646d36bee81ca594155be95002a9a7c743550ab1edd83b20fe4","observation":{"variables":["conductivity","oxygen","stir_rate","titer","turbidity","viscosity"],"target":"titer","candidates":["conductivity","oxygen","stir_rate","turbidity","viscosity"],"intervention_range":{"turbidity":[34.2,40.4],"conductivity":[16.7,27.1],"titer":[24.5,40.9],"stir_rate":[9.1,15.9],"viscosity":[17.9,30.3],"oxygen":[6.4,13.0]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on titer has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"direct_causes","instruction":"find the set of direct causes of titer: the variables that appear with a nonzero coefficient in titer's own structural equation (not causes acting only through other variables, not effects of titer, not merely correlated variables)","report":["causes"],"success_rule":"the COMMIT succeeds only if the committed set equals the set of direct causes exactly; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 5 interventions. Then: COMMIT causes=<comma-separated variable names> (or COMMIT causes=none).","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[{"kind":"observational","n":200,"mean":{"conductivity":21.92,"oxygen":9.8,"stir_rate":12.62,"titer":32.58,"turbidity":37.3,"viscosity":24.08},"cov":[[2.435,0.04916,-0.1599,-2.443,1.04,0.669],[0.04916,0.8956,0.009961,0.1197,-0.04825,-0.1418],[-0.1599,0.009961,0.9808,0.01634,-0.03937,-0.8527],[-2.443,0.1197,0.01634,6.304,-0.973,-2.104],[1.04,-0.04825,-0.03937,-0.973,0.8625,0.2004],[0.669,-0.1418,-0.8527,-2.104,0.2004,2.877]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":4},"hidden_truth":{"question":{"kind":"direct_causes"},"direct_causes_of_each_variable":{"conductivity":["turbidity"],"oxygen":[],"stir_rate":[],"titer":["conductivity"],"turbidity":[],"viscosity":["conductivity","stir_rate","titer"]},"variables_driven_by_the_unobserved_common_cause":["titer","viscosity"],"answer":["conductivity"]},"gold_commit":"COMMIT causes=conductivity","gold_commit_verified":true,"truth_sha256":"7ca7c0e525464c2381026d1982f7bd63bc61588c60171bb8b1571ebda2e29a7d"}
{"family":"causal_intervention","level":2,"split":"dev","seed":900002,"instance_hash":"d466a33a3d81702f11ef888ab9fc6d8dcd33a5bf65fd0ccc3f3d7e7f8aad28f1","observation":{"variables":["foam_height","humidity","oxygen","pigment","purity","redox","turbidity"],"target":"purity","candidates":["foam_height","humidity","oxygen","pigment","redox","turbidity"],"intervention_range":{"foam_height":[23.6,37.2],"turbidity":[22.6,32.8],"pigment":[22.8,37.0],"purity":[12.4,40.4],"humidity":[29.7,48.9],"redox":[-0.4,12.6],"oxygen":[10.1,30.7]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on purity has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"direct_causes","instruction":"find the set of direct causes of purity: the variables that appear with a nonzero coefficient in purity's own structural equation (not causes acting only through other variables, not effects of purity, not merely correlated variables)","report":["causes"],"success_rule":"the COMMIT succeeds only if the committed set equals the set of direct causes exactly; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 6 interventions. Then: COMMIT causes=<comma-separated variable names> (or COMMIT causes=none).","experiment_budget":{"maximum":6,"used":0,"remaining":6},"history":[{"kind":"observational","n":200,"mean":{"foam_height":30.26,"humidity":39.55,"oxygen":20.12,"pigment":29.98,"purity":26.11,"redox":6.26,"turbidity":27.58},"cov":[[4.119,-4.919,4.687,-3.66,7.665,-3.015,2.336],[-4.919,9.195,-5.873,4.686,-11.38,3.537,-3.546],[4.687,-5.873,8.332,-4.158,8.688,-4.633,3.169],[-3.66,4.686,-4.158,3.87,-7.647,2.635,-2.127],[7.665,-11.38,8.688,-7.647,17.38,-5.386,4.5],[-3.015,3.537,-4.633,2.635,-5.386,3.265,-1.75],[2.336,-3.546,3.169,-2.127,4.5,-1.75,2.177]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":9,"python_runs":4},"hidden_truth":{"question":{"kind":"direct_causes"},"direct_causes_of_each_variable":{"foam_height":[],"humidity":["pigment","purity","turbidity"],"oxygen":["redox","turbidity"],"pigment":["foam_height"],"purity":["pigment"],"redox":["foam_height"],"turbidity":["foam_height"]},"variables_driven_by_the_unobserved_common_cause":["foam_height","humidity","purity"],"answer":["pigment"]},"gold_commit":"COMMIT causes=pigment","gold_commit_verified":true,"truth_sha256":"80e63e615cd1fcac5700e3ca1dd49a3e94b200f489601f6a75c00c3088b44f7c"}
{"family":"causal_intervention","level":2,"split":"dev","seed":900003,"instance_hash":"a8a8965cc09950632ab3fed399105b2130fbbf66a5304485b4ea31cf1c288f34","observation":{"variables":["catalyst","conductivity","feed_flow","humidity","purity","turbidity","viscosity"],"target":"purity","candidates":["catalyst","conductivity","feed_flow","humidity","turbidity","viscosity"],"intervention_range":{"conductivity":[20.9,28.9],"catalyst":[4.1,16.7],"turbidity":[15.7,24.9],"viscosity":[18.6,26.8],"humidity":[24.3,39.7],"feed_flow":[13.2,32.4],"purity":[4.3,41.3]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on purity has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"direct_causes","instruction":"find the set of direct causes of purity: the variables that appear with a nonzero coefficient in purity's own structural equation (not causes acting only through other variables, not effects of purity, not merely correlated variables)","report":["causes"],"success_rule":"the COMMIT succeeds only if the committed set equals the set of direct causes exactly; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 6 interventions. Then: COMMIT causes=<comma-separated variable names> (or COMMIT causes=none).","experiment_budget":{"maximum":6,"used":0,"remaining":6},"history":[{"kind":"observational","n":200,"mean":{"catalyst":10.13,"conductivity":24.89,"feed_flow":23.02,"humidity":31.83,"purity":22.25,"turbidity":20.16,"viscosity":22.89},"cov":[[3.285,-1.653,-0.6275,0.6237,1.964,0.6178,-0.3731],[-1.653,1.33,-0.11,0.08521,0.4165,0.08423,-0.2958],[-0.6275,-0.11,7.529,-5.622,-12.96,-2.171,0.3208],[0.6237,0.08521,-5.622,4.819,10.32,1.853,-0.3216],[1.964,0.4165,-12.96,10.32,26.39,4.775,-2.205],[0.6178,0.08423,-2.171,1.853,4.775,1.613,-0.3422],[-0.3731,-0.2958,0.3208,-0.3216,-2.205,-0.3422,1.538]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":9,"python_runs":4},"hidden_truth":{"question":{"kind":"direct_causes"},"direct_causes_of_each_variable":{"catalyst":["conductivity"],"conductivity":[],"feed_flow":["humidity"],"humidity":["turbidity"],"purity":["feed_flow","turbidity","viscosity"],"turbidity":["catalyst","conductivity"],"viscosity":["catalyst","conductivity"]},"variables_driven_by_the_unobserved_common_cause":["humidity","purity"],"answer":["feed_flow","turbidity","viscosity"]},"gold_commit":"COMMIT causes=feed_flow,turbidity,viscosity","gold_commit_verified":true,"truth_sha256":"dc959f9c00b0ce0f38eb4ec0bef9e69bd9d956210414011d2acf998492fba27a"}
{"family":"causal_intervention","level":2,"split":"dev","seed":900004,"instance_hash":"87871d04fd770dbaf2f78e4bf10394187443348e9daeaa73e8d6b90b43c85366","observation":{"variables":["catalyst","conductivity","light_dose","output","pressure","viscosity"],"target":"output","candidates":["catalyst","conductivity","light_dose","pressure","viscosity"],"intervention_range":{"pressure":[12.2,25.6],"viscosity":[32.8,40.0],"light_dose":[6.9,12.5],"conductivity":[15.5,24.5],"catalyst":[28.2,44.6],"output":[5.2,38.6]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on output has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"direct_causes","instruction":"find the set of direct causes of output: the variables that appear with a nonzero coefficient in output's own structural equation (not causes acting only through other variables, not effects of output, not merely correlated variables)","report":["causes"],"success_rule":"the COMMIT succeeds only if the committed set equals the set of direct causes exactly; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 5 interventions. Then: COMMIT causes=<comma-separated variable names> (or COMMIT causes=none).","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[{"kind":"observational","n":200,"mean":{"catalyst":36.01,"conductivity":19.88,"light_dose":9.76,"output":22.63,"pressure":19.05,"viscosity":36.41},"cov":[[4.901,1.02,-0.7108,-8.146,-3.417,0.0176],[1.02,1.86,-0.0556,-1.424,-0.8419,-0.04913],[-0.7108,-0.0556,0.5871,0.3158,0.1277,-0.01164],[-8.146,-1.424,0.3158,18.06,6.752,1.006],[-3.417,-0.8419,0.1277,6.752,3.37,-0.05836],[0.0176,-0.04913,-0.01164,1.006,-0.05836,0.8247]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":4},"hidden_truth":{"question":{"kind":"direct_causes"},"direct_causes_of_each_variable":{"catalyst":["light_dose","pressure"],"conductivity":["pressure"],"light_dose":[],"output":["catalyst","light_dose","viscosity"],"pressure":[],"viscosity":[]},"variables_driven_by_the_unobserved_common_cause":["conductivity","output","pressure"],"answer":["catalyst","light_dose","viscosity"]},"gold_commit":"COMMIT causes=catalyst,light_dose,viscosity","gold_commit_verified":true,"truth_sha256":"761f2d4aa68c9148a9de9ce54465f1e63ba1c3d7fd7eecbb84eb6789f4a45ec5"}
{"family":"causal_intervention","level":2,"split":"dev","seed":900005,"instance_hash":"7d55a6cfef70e5f8c62c8af2ef966dae59662e9fd02080a6e1da446b3b22b348","observation":{"variables":["catalyst","growth","oxygen","redox","turbidity","viscosity"],"target":"growth","candidates":["catalyst","oxygen","redox","turbidity","viscosity"],"intervention_range":{"redox":[-0.5,12.9],"oxygen":[11.4,27.6],"growth":[27.3,39.1],"viscosity":[23.1,29.9],"catalyst":[3.2,30.8],"turbidity":[30.7,42.7]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on growth has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"joint_effect","do":{"oxygen":13.3,"redox":1.0},"tolerance":1.6,"instruction":"predict the mean of growth under the joint intervention do(oxygen=13.3, redox=1.0), both variables set at once; the laboratory can set only one variable per batch","report":["value"],"success_rule":"the COMMIT succeeds only if |value - true mean of growth under do(oxygen=13.3, redox=1.0)| <= 1.6; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 2 interventions. Then: COMMIT value=<number> (the predicted mean of growth).","experiment_budget":{"maximum":2,"used":0,"remaining":2},"history":[{"kind":"observational","n":200,"mean":{"catalyst":17.07,"growth":33.34,"oxygen":19.48,"redox":6.2,"turbidity":36.78,"viscosity":26.5},"cov":[[17.53,6.106,9.302,-6.306,-5.853,-0.1479],[6.106,3.22,2.696,-1.649,-1.655,-0.1247],[9.302,2.696,5.823,-4.199,-3.633,-0.0186],[-6.306,-1.649,-4.199,4.038,2.604,0.005051],[-5.853,-1.655,-3.633,2.604,2.949,-0.01036],[-0.1479,-0.1247,-0.0186,0.005051,-0.01036,0.9765]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":6,"python_runs":4},"hidden_truth":{"question":{"kind":"joint_effect","do":{"oxygen":13.3,"redox":1.0},"tolerance":1.6},"direct_causes_of_each_variable":{"catalyst":["growth","oxygen"],"growth":["oxygen","redox"],"oxygen":["redox"],"redox":[],"turbidity":["oxygen"],"viscosity":[]},"variables_driven_by_the_unobserved_common_cause":["growth","redox"],"answer":23.546},"gold_commit":"COMMIT value=23.546","gold_commit_verified":true,"truth_sha256":"981f37eb54911730ce10f21f624c29e828a8c49fc5708819130a63e74494338b"}
{"family":"causal_intervention","level":2,"split":"dev","seed":900006,"instance_hash":"cd4d2156b0b2294f1e42c70259ada3f36fab6561e645939a6baf98364a087a8a","observation":{"variables":["growth","ph_level","pressure","redox","stir_rate","viscosity"],"target":"growth","candidates":["ph_level","pressure","redox","stir_rate","viscosity"],"intervention_range":{"redox":[2.3,10.1],"pressure":[28.4,38.2],"growth":[29.0,45.6],"ph_level":[13.0,25.6],"stir_rate":[11.5,22.1],"viscosity":[9.2,27.0]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on growth has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"direct_causes","instruction":"find the set of direct causes of growth: the variables that appear with a nonzero coefficient in growth's own structural equation (not causes acting only through other variables, not effects of growth, not merely correlated variables)","report":["causes"],"success_rule":"the COMMIT succeeds only if the committed set equals the set of direct causes exactly; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 5 interventions. Then: COMMIT causes=<comma-separated variable names> (or COMMIT causes=none).","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[{"kind":"observational","n":200,"mean":{"growth":37.17,"ph_level":19.5,"pressure":33.3,"redox":6.18,"stir_rate":16.67,"viscosity":18.34},"cov":[[5.916,-1.974,2.383,-1.156,-1.305,-4.361],[-1.974,3.619,0.1078,-0.2652,-0.07009,3.979],[2.383,0.1078,1.934,-0.9621,-0.7295,-1.706],[-1.156,-0.2652,-0.9621,1.347,-0.4509,0.632],[-1.305,-0.07009,-0.7295,-0.4509,2.648,0.6415],[-4.361,3.979,-1.706,0.632,0.6415,6.695]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":4},"hidden_truth":{"question":{"kind":"direct_causes"},"direct_causes_of_each_variable":{"growth":["pressure"],"ph_level":[],"pressure":["redox"],"redox":[],"stir_rate":["growth","redox"],"viscosity":["ph_level","pressure"]},"variables_driven_by_the_unobserved_common_cause":["growth","ph_level","stir_rate"],"answer":["pressure"]},"gold_commit":"COMMIT causes=pressure","gold_commit_verified":true,"truth_sha256":"300626c299b8c3fba271c991a4cb45c5a827fd5bfd506f920dbd2ecbaeff155f"}
{"family":"causal_intervention","level":2,"split":"dev","seed":900007,"instance_hash":"626a17be252802af157c1619d37635fe82945549688b76d46e8cbc718b6187e9","observation":{"variables":["catalyst","foam_height","light_dose","oxygen","ph_level","pressure","yield"],"target":"yield","candidates":["catalyst","foam_height","light_dose","oxygen","ph_level","pressure"],"intervention_range":{"ph_level":[36.1,42.1],"catalyst":[15.6,29.6],"yield":[13.1,35.1],"pressure":[28.7,40.9],"oxygen":[16.1,22.7],"light_dose":[20.5,39.5],"foam_height":[7.6,24.8]},"model":"hidden linear structural causal model over the observed variables: each variable = intercept + sum of coefficient x (its direct causes) + Gaussian noise; the causal graph is a random DAG (names carry no information); an unobserved common cause may drive several variables; every direct effect on yield has |coefficient| >= 0.6. INTERVENE X=v sets X to v in a fresh batch of 20 samples (X's own causes are cut; the other variables respond through the graph); the observational record has 200 samples, cov rows/columns in the order of 'variables'","objective":{"kind":"joint_effect","do":{"catalyst":28.0,"ph_level":37.3},"tolerance":1.15,"instruction":"predict the mean of yield under the joint intervention do(catalyst=28.0, ph_level=37.3), both variables set at once; the laboratory can set only one variable per batch","report":["value"],"success_rule":"the COMMIT succeeds only if |value - true mean of yield under do(catalyst=28.0, ph_level=37.3)| <= 1.15; a malformed action, an unknown variable, a value outside the intervention range or an intervention beyond the budget ends the episode at zero"},"action_grammar":"INTERVENE <variable>=<value> (one observed variable, value inside its intervention_range); at most 2 interventions. Then: COMMIT value=<number> (the predicted mean of yield).","experiment_budget":{"maximum":2,"used":0,"remaining":2},"history":[{"kind":"observational","n":200,"mean":{"catalyst":22.49,"foam_height":16.48,"light_dose":29.9,"oxygen":19.49,"ph_level":39.05,"pressure":34.8,"yield":24.04},"cov":[[3.956,0.2612,2.454,-0.02475,-0.6486,-2.598,4.597],[0.2612,7.536,5.287,1.146,-1.46,-0.1954,5.198],[2.454,5.287,7.995,-0.03486,-1.795,-0.7654,7.506],[-0.02475,1.146,-0.03486,0.9741,0.008367,-0.0592,0.08095],[-0.6486,-1.46,-1.795,0.008367,0.7787,0.3929,-2.135],[-2.598,-0.1954,-0.7654,-0.0592,0.3929,2.752,-2.964],[4.597,5.198,7.506,0.08095,-2.135,-2.964,9.833]],"supplied":true}],"status":"active","world":"causal_intervention_bench"},"turn_limits":{"lab_action_turns":6,"python_runs":4},"hidden_truth":{"question":{"kind":"joint_effect","do":{"catalyst":28.0,"ph_level":37.3},"tolerance":1.15},"direct_causes_of_each_variable":{"catalyst":["ph_level"],"foam_height":["catalyst","oxygen","yield"],"light_dose":["pressure","yield"],"oxygen":[],"ph_level":[],"pressure":["catalyst"],"yield":["catalyst","ph_level"]},"variables_driven_by_the_unobserved_common_cause":["catalyst","yield"],"answer":34.702},"gold_commit":"COMMIT value=34.702","gold_commit_verified":true,"truth_sha256":"9a62f00ba610942d6bf71fecee38ae3debd00f3a09d5d350eec497c164621d56"}
{"family":"trace_debugging","level":0,"split":"dev","seed":900000,"instance_hash":"58b7c9be7e5e192cbc28be70e85323423f975bee355d08c3bd68b1493ec5f8c6","observation":{"program":["def f(c, w):"," 1|  bonus = 0"," 2|  gap = 0"," 3|  for j from 2 to c + 3:"," 4|      if j % 3 == 0:"," 5|          gap = gap - c - c"," 6|  gap = gap + w * bonus"," 7|  return gap + bonus * 2"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(c, w) with every argument an integer from 0 to 6","examples":["f(0, 1) = 0","f(1, 2) = -2","f(1, 3) = -2","f(2, 5) = -4","f(4, 1) = -16","f(6, 0) = -24"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":3,"bug_kind":"constant","shown_line":"for j from 2 to c + 3:","intended_expression":"2 to c + 2"},"gold_commit":"COMMIT line=3 fix=2 to c + 2","gold_commit_verified":true,"truth_sha256":"d6adac8956704e07242c7fc185debac3b600518edc8f3521f5267a0fd3678d90"}
{"family":"trace_debugging","level":0,"split":"dev","seed":900001,"instance_hash":"672c1025e7a8135481a35b5ad1504f92f9840edeb55f58c4cef0c734c805aa5e","observation":{"program":["def f(b, a):"," 1|  score = b"," 2|  step = score * 2"," 3|  for j from 1 to b:"," 4|      if score < a:"," 5|          step = step + score"," 6|      step = a - step"," 7|  score = score - (step + 3)"," 8|  return score"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(b, a) with every argument an integer from 0 to 6","examples":["f(0, 1) = -3","f(1, 4) = 0","f(2, 2) = -6","f(3, 4) = 7","f(5, 2) = 9","f(5, 3) = 9"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":6,"bug_kind":"variable","shown_line":"step = a - step","intended_expression":"j - step"},"gold_commit":"COMMIT line=6 fix=j - step","gold_commit_verified":true,"truth_sha256":"e7708bcd25156ecc7bcbc0ce863b93bdaa577643b2d435193a3f19fbcba63dc1"}
{"family":"trace_debugging","level":0,"split":"dev","seed":900002,"instance_hash":"ad674bee8a8a1092b59ef6092d7eac82cb9bdf5a75694488601c4e99241ed6b6","observation":{"program":["def f(w, b):"," 1|  gap = b"," 2|  total = gap * 1"," 3|  for j from 0 to total - 1:"," 4|      total = total + gap * 3"," 5|  return total + gap * 2"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(w, b) with every argument an integer from 0 to 6","examples":["f(0, 1) = 3","f(0, 6) = 18","f(1, 0) = 0","f(1, 4) = 24","f(4, 2) = 30","f(6, 1) = 21"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":3,"bug_kind":"variable","shown_line":"for j from 0 to total - 1:","intended_expression":"0 to w - 1"},"gold_commit":"COMMIT line=3 fix=0 to w - 1","gold_commit_verified":true,"truth_sha256":"0c7e04d88f67abfe99fe45b96bc4b33d5fb2c4ef67b7786bbc99c731aeee7a41"}
{"family":"trace_debugging","level":0,"split":"dev","seed":900003,"instance_hash":"d13ca48a5d23280256e4ecf1d05f29ed0c9039c62c3e383d3df166ba55454871","observation":{"program":["def f(a, c):"," 1|  gap = a"," 2|  carry = 2"," 3|  for i from 2 to a + 2:"," 4|      if gap % 2 == 0:"," 5|          gap = carry - gap"," 6|      else:"," 7|          carry = carry - gap + i"," 8|  return gap + carry * 3"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(a, c) with every argument an integer from 0 to 6","examples":["f(0, 5) = 8","f(1, 2) = 16","f(1, 4) = 16","f(3, 0) = -36","f(3, 1) = -36","f(3, 2) = -36"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":7,"bug_kind":"variable","shown_line":"carry = carry - gap + i","intended_expression":"carry - gap + carry"},"gold_commit":"COMMIT line=7 fix=carry - gap + carry","gold_commit_verified":true,"truth_sha256":"21041dae1bde05c271fc30fab440c9771f4cd0f608a8d2954e220dd7cd0f5f8c"}
{"family":"trace_debugging","level":0,"split":"dev","seed":900004,"instance_hash":"302f332a6c77f46c39e6cdcb5ba64bafcefb0fd008cb620baa3b09c1c861653b","observation":{"program":["def f(c, p):"," 1|  score = c"," 2|  prev = 2"," 3|  for i from 2 to p:"," 4|      if prev >= p:"," 5|          score = prev * 1"," 6|      else:"," 7|          prev = prev + c - c"," 8|  return prev + score * 3"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(c, p) with every argument an integer from 0 to 6","examples":["f(1, 2) = 8","f(1, 3) = 8","f(4, 1) = 14","f(6, 1) = 20","f(6, 2) = 20","f(6, 6) = 20"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":4,"bug_kind":"variable","shown_line":"if prev >= p:","intended_expression":"prev >= c"},"gold_commit":"COMMIT line=4 fix=prev >= c","gold_commit_verified":true,"truth_sha256":"eb8f89fa9af22cd8b5631d5f8d301b0cf4be795372add5d794a5fd1ac0c35dd8"}
{"family":"trace_debugging","level":0,"split":"dev","seed":900005,"instance_hash":"8598028c08c9259c51804351112ba448c5ecaaa3df64c18e0f6fa68d4dfc22e4","observation":{"program":["def f(w, s):"," 1|  gap = s + 1"," 2|  best = 2"," 3|  for i from 0 to w + 1:"," 4|      if best > s:"," 5|          best = best + i * w"," 6|      else:"," 7|          best = s - best"," 8|      gap = gap - w - i"," 9|  gap = gap + w * s","10|  return best - gap"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(w, s) with every argument an integer from 0 to 6","examples":["f(1, 6) = -8","f(2, 1) = 24","f(3, 1) = 52","f(3, 4) = 9","f(6, 1) = 238","f(6, 2) = 233"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":7,"bug_kind":"variable","shown_line":"best = s - best","intended_expression":"w - best"},"gold_commit":"COMMIT line=7 fix=w - best","gold_commit_verified":true,"truth_sha256":"4f4d4a10be6b12deb4f79aebfe60b5bcc6a1e77f1cba4337eb1f6495a2f4d874"}
{"family":"trace_debugging","level":0,"split":"dev","seed":900006,"instance_hash":"792113db85a937647e1d569ae90f4640773b69b6ef23b4fe8c2d6eff13ffc3a3","observation":{"program":["def f(a, b):"," 1|  cur = b - 3"," 2|  tot = 2"," 3|  for j from 2 to b * 1:"," 4|      if j % 3 == 1:"," 5|          tot = tot + cur"," 6|      else:"," 7|          tot = tot + b"," 8|      tot = tot + a * 3"," 9|  return tot"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(a, b) with every argument an integer from 0 to 6","examples":["f(0, 0) = 2","f(1, 0) = 2","f(1, 6) = 35","f(2, 2) = 2","f(4, 2) = 2","f(5, 4) = 40"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":3,"bug_kind":"operator","shown_line":"for j from 2 to b * 1:","intended_expression":"2 to b - 1"},"gold_commit":"COMMIT line=3 fix=2 to b - 1","gold_commit_verified":true,"truth_sha256":"fd15053b87ef87503287e122c811215e368d15fe81272e2f972911c71dbecba3"}
{"family":"trace_debugging","level":0,"split":"dev","seed":900007,"instance_hash":"304c1975b564faa3092e14b3fbf9ad1cf42a0e83e4d83ccc3836f15bad1944e2","observation":{"program":["def f(m, s):"," 1|  bonus = s - 3"," 2|  low = 1"," 3|  for i from 3 to m + 1:"," 4|      bonus = i - 3"," 5|  return low * bonus"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(m, s) with every argument an integer from 0 to 6","examples":["f(1, 2) = -1","f(1, 3) = -1","f(2, 1) = 0","f(4, 5) = 2","f(6, 4) = 4","f(6, 6) = 4"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":3,"bug_kind":"constant","shown_line":"for i from 3 to m + 1:","intended_expression":"2 to m + 1"},"gold_commit":"COMMIT line=3 fix=2 to m + 1","gold_commit_verified":true,"truth_sha256":"cac6b302eb888ac7b09164a4e158bee30fc0c11d8c47838d6e02fe71dea26737"}
{"family":"trace_debugging","level":1,"split":"dev","seed":900000,"instance_hash":"f74798e038c56da4e753889546b90ad47446a0d707b8fcd847b9423f31c5ccc3","observation":{"program":["def f(p, w):"," 1|  score = p"," 2|  prev = w - 3"," 3|  acc = prev * 2"," 4|  for i from 2 to w:"," 5|      if w % 2 != 0:"," 6|          score = p - score"," 7|      else:"," 8|          score = acc + 3"," 9|      acc = p - acc","10|  acc = acc - score * 3","11|  return acc"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(p, w) with every argument an integer from 0 to 6","examples":["f(1, 4) = -16","f(3, 3) = 0","f(3, 4) = -14","f(3, 5) = 16","f(5, 5) = 10","f(6, 5) = 7"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":5,"bug_kind":"variable","shown_line":"if w % 2 != 0:","intended_expression":"i % 2 != 0"},"gold_commit":"COMMIT line=5 fix=i % 2 != 0","gold_commit_verified":true,"truth_sha256":"f5f67c7b08d92096906b353073e141f0f62934d6ec6f03a4948a9b58b62fa058"}
{"family":"trace_debugging","level":1,"split":"dev","seed":900001,"instance_hash":"fb74f8e8577291dacdc4cb2c307f15942b00ea999baf8ee3e4f88df8867c890b","observation":{"program":["def f(n, c):"," 1|  carry = 1"," 2|  prev = c + 1"," 3|  tot = c"," 4|  for j from 0 to 4:"," 5|      if carry % 2 == 0:"," 6|          prev = carry - prev"," 7|      else:"," 8|          tot = tot + j * prev"," 9|      carry = carry + prev","10|  carry = prev + c","11|  return tot + carry"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(n, c) with every argument an integer from 0 to 6","examples":["f(1, 2) = 15","f(1, 5) = 76","f(2, 2) = 15","f(4, 0) = 7","f(4, 2) = 15","f(4, 5) = 76"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":4,"bug_kind":"constant","shown_line":"for j from 0 to 4:","intended_expression":"1 to 4"},"gold_commit":"COMMIT line=4 fix=1 to 4","gold_commit_verified":true,"truth_sha256":"42bf4385f938521626d7ffae7bae4633c87037edbf907137ce7881d6bf6754e1"}
{"family":"trace_debugging","level":1,"split":"dev","seed":900002,"instance_hash":"be6145f15ddc32eb3aca4261aa5fe9ecbb9fe6b1e41464c744476aaaed314860","observation":{"program":["def f(m, r):"," 1|  high = r"," 2|  bonus = 1"," 3|  gap = 2"," 4|  for i from 2 to m:"," 5|      if i % 2 != 1:"," 6|          high = high + bonus * 3"," 7|      else:"," 8|          bonus = high * 3"," 9|      gap = gap + i * m","10|  high = bonus + 1","11|  return gap - high"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(m, r) with every argument an integer from 0 to 6","examples":["f(1, 3) = 0","f(1, 5) = 0","f(3, 4) = -2","f(5, 1) = 8","f(5, 3) = -34","f(6, 5) = -26"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":6,"bug_kind":"constant","shown_line":"high = high + bonus * 3","intended_expression":"high + bonus * 2"},"gold_commit":"COMMIT line=6 fix=high + bonus * 2","gold_commit_verified":true,"truth_sha256":"13cddf941049316ab11aeef0a6271a2fc1d533d0b04a23d40b729868fc58eecb"}
{"family":"trace_debugging","level":1,"split":"dev","seed":900003,"instance_hash":"265422e8b688b67c4e878d3b4396724cd25e2b8daf91711b8c7a46bf3be49057","observation":{"program":["def f(n, b):"," 1|  step = n"," 2|  acc = step * 2"," 3|  cur = acc + 2"," 4|  for i from 2 to n:"," 5|      if cur % 2 == 0:"," 6|          cur = i + cur"," 7|      else:"," 8|          step = step - n + i"," 9|      cur = cur + n * 3","10|  cur = cur - step * n","11|  return cur"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(n, b) with every argument an integer from 0 to 6","examples":["f(0, 3) = 2","f(1, 6) = 3","f(2, 1) = -2","f(2, 2) = -2","f(4, 4) = 7","f(6, 3) = 51"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":6,"bug_kind":"operator","shown_line":"cur = i + cur","intended_expression":"i - cur"},"gold_commit":"COMMIT line=6 fix=i - cur","gold_commit_verified":true,"truth_sha256":"89482a1aa267df2a8381e2a4a05d1d58b9453237ec4674e75162a725b96240a6"}
{"family":"trace_debugging","level":1,"split":"dev","seed":900004,"instance_hash":"235bd36e029f111dd15a71bcaa189ae9c80e11e7e3d91d0c979e06e79b4ff32f","observation":{"program":["def f(q, m):"," 1|  step = 2"," 2|  bonus = q"," 3|  total = 0"," 4|  for j from 0 to q + 1:"," 5|      if j <= m:"," 6|          total = total - bonus + step"," 7|      else:"," 8|          bonus = bonus + step * 2"," 9|      total = q - total","10|  step = bonus * total","11|  return bonus - step * 3"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(q, m) with every argument an integer from 0 to 6","examples":["f(0, 3) = 0","f(0, 5) = 0","f(3, 0) = -154","f(4, 0) = 140","f(5, 0) = -442","f(6, 5) = 10"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":1,"bug_kind":"constant","shown_line":"step = 2","intended_expression":"1"},"gold_commit":"COMMIT line=1 fix=1","gold_commit_verified":true,"truth_sha256":"2c9c1c79cb8554e33809b09c38481d56db086e330be010d06cff33f87a8e58e4"}
{"family":"trace_debugging","level":1,"split":"dev","seed":900005,"instance_hash":"1851db9fb6381a7a34e6d288e0045d999d08abb66e76d56c330a6fc826123e7d","observation":{"program":["def f(b, n):"," 1|  best = b + 2"," 2|  cnt = 1"," 3|  mark = 0"," 4|  for i from 2 to n + 1:"," 5|      if i >= n:"," 6|          mark = mark + (i + 2)"," 7|      else:"," 8|          mark = mark + n * 2"," 9|      cnt = cnt + i * best","10|  mark = mark - cnt * n","11|  return mark - best"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(b, n) with every argument an integer from 0 to 6","examples":["f(0, 5) = -155","f(1, 0) = -3","f(2, 2) = -36","f(2, 5) = -357","f(3, 3) = -123","f(6, 4) = -426"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":6,"bug_kind":"operator","shown_line":"mark = mark + (i + 2)","intended_expression":"mark + i * 2"},"gold_commit":"COMMIT line=6 fix=mark + i * 2","gold_commit_verified":true,"truth_sha256":"64c4937266b86aa7d5832a474e2ae7a4ab812545b072df745d8cf5b4b038ad80"}
{"family":"trace_debugging","level":1,"split":"dev","seed":900006,"instance_hash":"894e09560005dab413435c12b18685bef64997618a773a7205d6b83387c7627c","observation":{"program":["def f(s, c):"," 1|  step = 3"," 2|  level = s - 1"," 3|  cur = step + 1"," 4|  for j from 1 to c:"," 5|      if j >= s:"," 6|          step = cur - step"," 7|      cur = cur + j * j"," 8|  step = step - (level - 2)"," 9|  return step - level * 2"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(s, c) with every argument an integer from 0 to 6","examples":["f(0, 4) = 17","f(1, 3) = 7","f(1, 4) = 14","f(4, 5) = 11","f(5, 0) = -8","f(5, 5) = 21"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":1,"bug_kind":"constant","shown_line":"step = 3","intended_expression":"2"},"gold_commit":"COMMIT line=1 fix=2","gold_commit_verified":true,"truth_sha256":"12a500d4ed85715f208a7d8e79792df3d04592de6a69d087e60622de90ce0eaf"}
{"family":"trace_debugging","level":1,"split":"dev","seed":900007,"instance_hash":"cc145ed91eba1ae18925be202d77ab34f0cd2afc620717e613b5c96bd5093c1d","observation":{"program":["def f(r, n):"," 1|  high = 2"," 2|  total = n - 2"," 3|  level = r"," 4|  for j from 0 to n:"," 5|      if j % 2 == 0:"," 6|          total = total + j * level"," 7|      else:"," 8|          total = r - 1"," 9|      level = level + j * n","10|  total = total - (level - 2)","11|  return high + total"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(r, n) with every argument an integer from 0 to 6","examples":["f(0, 3) = -11","f(0, 6) = 242","f(2, 2) = 4","f(2, 5) = -52","f(5, 0) = -3","f(6, 2) = 12"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>) or TRACE input=(<int>, <int>) line=<n>; at most 5 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":5,"used":0,"remaining":5},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":8,"python_runs":0},"hidden_truth":{"bug_line":9,"bug_kind":"variable","shown_line":"level = level + j * n","intended_expression":"level + j * j"},"gold_commit":"COMMIT line=9 fix=level + j * j","gold_commit_verified":true,"truth_sha256":"a05fe30e97da378ee9ea55ed25801e1af5d95aec142a3bb11dda16a14bfed9e4"}
{"family":"trace_debugging","level":2,"split":"dev","seed":900000,"instance_hash":"6abbe1f90d7c0d70371220053390a5cf5108d1872d0f6f56034ae33691bffe9e","observation":{"program":["def f(s, n, p):"," 1|  low = p"," 2|  score = n"," 3|  prev = low + 2"," 4|  for j from 0 to s:"," 5|      if j <= 4:"," 6|          score = p - score"," 7|      else:"," 8|          score = score - prev - low"," 9|      low = low - n","10|  prev = prev + (low + p)","11|  return score * prev"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(s, n, p) with every argument an integer from 0 to 5","examples":["f(0, 0, 5) = 85","f(0, 1, 0) = -1","f(0, 1, 5) = 64","f(0, 3, 5) = 28","f(3, 3, 5) = -15","f(3, 4, 2) = 32","f(4, 1, 0) = 9"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>, <int>) or TRACE input=(<int>, <int>, <int>) line=<n>; at most 4 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":4,"used":0,"remaining":4},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":7,"python_runs":0},"hidden_truth":{"bug_line":6,"bug_kind":"variable","shown_line":"score = p - score","intended_expression":"low - score"},"gold_commit":"COMMIT line=6 fix=low - score","gold_commit_verified":true,"truth_sha256":"8a3ba0e0a1d92d078924e674afab0fb346b04e1eb21752313ac5ad99aadc44da"}
{"family":"trace_debugging","level":2,"split":"dev","seed":900001,"instance_hash":"1ebd02bd91e39f060c649a50b989517f6624b67394c77c3795e963590ed6a047","observation":{"program":["def f(q, r, c):"," 1|  gap = q"," 2|  run = gap * 2"," 3|  mark = c + 1"," 4|  for i from 1 to 4:"," 5|      if run > 3:"," 6|          mark = mark + i * i"," 7|      else:"," 8|          run = run + mark * 2"," 9|      gap = gap + i * 3","10|      run = i - run","11|  run = run + (gap - r)","12|  return run * gap"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(q, r, c) with every argument an integer from 0 to 5","examples":["f(0, 5, 0) = 810","f(1, 3, 2) = 992","f(2, 5, 0) = 864","f(3, 1, 3) = 825","f(3, 4, 4) = 627","f(4, 2, 4) = 816","f(5, 2, 5) = 840"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>, <int>) or TRACE input=(<int>, <int>, <int>) line=<n>; at most 4 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":4,"used":0,"remaining":4},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":7,"python_runs":0},"hidden_truth":{"bug_line":8,"bug_kind":"constant","shown_line":"run = run + mark * 2","intended_expression":"run + mark * 3"},"gold_commit":"COMMIT line=8 fix=run + mark * 3","gold_commit_verified":true,"truth_sha256":"a4a90c7ab125e2e8300aab20b7be88e05ab0651db374794e4ec2ee53ab670f62"}
{"family":"trace_debugging","level":2,"split":"dev","seed":900002,"instance_hash":"7551d17073c341021e7d4d7dd5e0397b9247328d316350715d7b377df6df0f5a","observation":{"program":["def f(m, n, q):"," 1|  cur = m - 3"," 2|  carry = q"," 3|  total = carry * 1"," 4|  for i from 2 to m:"," 5|      if i > 2:"," 6|          carry = carry - total + total"," 7|      else:"," 8|          cur = cur + total * m"," 9|      cur = cur - total + cur","10|      carry = carry - m","11|  total = m + q","12|  total = total + (carry + cur)","13|  return total - carry * 3"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(m, n, q) with every argument an integer from 0 to 5","examples":["f(0, 5, 1) = -4","f(1, 0, 4) = -5","f(1, 2, 2) = -3","f(1, 5, 2) = -3","f(1, 5, 4) = -5","f(3, 2, 2) = 31","f(5, 5, 4) = 173"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>, <int>) or TRACE input=(<int>, <int>, <int>) line=<n>; at most 4 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":4,"used":0,"remaining":4},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":7,"python_runs":0},"hidden_truth":{"bug_line":8,"bug_kind":"variable","shown_line":"cur = cur + total * m","intended_expression":"cur + i * m"},"gold_commit":"COMMIT line=8 fix=cur + i * m","gold_commit_verified":true,"truth_sha256":"e66fb4ca198e3c8072ceb191f105e5886802722eda5851949a490a1c5fdc4d11"}
{"family":"trace_debugging","level":2,"split":"dev","seed":900003,"instance_hash":"fe1523dbe1fad6baf9bc2096d3a45dc49240f9f217b294730a6f53d451f516ce","observation":{"program":["def f(a, r, b):"," 1|  mark = a + 1"," 2|  low = a + 2"," 3|  acc = a"," 4|  for i from 0 to b:"," 5|      if i % 2 != 1:"," 6|          low = low + r - acc"," 7|      else:"," 8|          low = low + r * 3"," 9|      mark = mark - low + i","10|  mark = mark + b * acc","11|  return acc + mark"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(a, r, b) with every argument an integer from 0 to 5","examples":["f(0, 2, 3) = -13","f(1, 2, 0) = -1","f(2, 2, 4) = -33","f(3, 5, 2) = -25","f(4, 0, 1) = -2","f(4, 5, 5) = -112","f(5, 2, 5) = -90"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>, <int>) or TRACE input=(<int>, <int>, <int>) line=<n>; at most 4 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":4,"used":0,"remaining":4},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":7,"python_runs":0},"hidden_truth":{"bug_line":8,"bug_kind":"variable","shown_line":"low = low + r * 3","intended_expression":"low + a * 3"},"gold_commit":"COMMIT line=8 fix=low + a * 3","gold_commit_verified":true,"truth_sha256":"dec93d582ec0733a345e8760d4bc69ba7b79bf5269714ff04493c3da6a876a90"}
{"family":"trace_debugging","level":2,"split":"dev","seed":900004,"instance_hash":"4c8595a61ad6759a315e97e65237e28db1a21de07b0b4f35e1809e6abfb34e9c","observation":{"program":["def f(p, s, w):"," 1|  prev = 0"," 2|  cnt = prev + 1"," 3|  level = prev * 2"," 4|  for j from 0 to cnt:"," 5|      if cnt > p:"," 6|          level = level - p"," 7|      else:"," 8|          level = p - 2"," 9|      level = cnt - level","10|      cnt = cnt - level + level","11|  prev = prev - p * cnt","12|  return level - prev * 3"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(p, s, w) with every argument an integer from 0 to 5","examples":["f(0, 1, 5) = 1","f(0, 4, 0) = 1","f(1, 4, 1) = 5","f(1, 4, 2) = 5","f(2, 3, 2) = 7","f(3, 3, 2) = 9","f(4, 3, 5) = 11"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>, <int>) or TRACE input=(<int>, <int>, <int>) line=<n>; at most 4 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":4,"used":0,"remaining":4},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":7,"python_runs":0},"hidden_truth":{"bug_line":4,"bug_kind":"variable","shown_line":"for j from 0 to cnt:","intended_expression":"0 to p"},"gold_commit":"COMMIT line=4 fix=0 to p","gold_commit_verified":true,"truth_sha256":"87ac4335ac9e595d0a3102f1b41d14c06b964b86c1123f0315430e567df3c453"}
{"family":"trace_debugging","level":2,"split":"dev","seed":900005,"instance_hash":"eb0e3dfbce68c7010a0984af12840aaac9c9101c9c4fc3e76e8b9f35dc49fb89","observation":{"program":["def f(q, w, b):"," 1|  high = 1"," 2|  bonus = 2"," 3|  mark = 2"," 4|  for i from 2 to 4:"," 5|      if bonus > q:"," 6|          mark = mark - i * w"," 7|      else:"," 8|          bonus = high - bonus"," 9|      mark = mark + bonus","10|  high = high - (mark - 2)","11|  return bonus - high"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(q, w, b) with every argument an integer from 0 to 5","examples":["f(0, 5, 3) = 52","f(2, 5, 5) = -2","f(3, 0, 0) = -2","f(3, 3, 4) = -2","f(4, 3, 1) = -2","f(5, 4, 4) = -2","f(5, 5, 0) = -2"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>, <int>) or TRACE input=(<int>, <int>, <int>) line=<n>; at most 4 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":4,"used":0,"remaining":4},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":7,"python_runs":0},"hidden_truth":{"bug_line":6,"bug_kind":"operator","shown_line":"mark = mark - i * w","intended_expression":"mark + i * w"},"gold_commit":"COMMIT line=6 fix=mark + i * w","gold_commit_verified":true,"truth_sha256":"c23ff5934a2de35620b82ab1519590d481bada42b900fb36fde37b429a49b48a"}
{"family":"trace_debugging","level":2,"split":"dev","seed":900006,"instance_hash":"e813296a940106783071a6fe5bde81df3c3c632a3d0f84b27fbebd145a134e51","observation":{"program":["def f(c, b, n):"," 1|  score = 2"," 2|  carry = n + 2"," 3|  mark = carry + 2"," 4|  for i from 0 to b:"," 5|      if i % 3 == 2:"," 6|          score = carry - b + score"," 7|      else:"," 8|          carry = carry - n"," 9|      mark = mark + i * b","10|  score = score - (b - n)","11|  score = score + (b - mark)","12|  return carry + score * 2"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(c, b, n) with every argument an integer from 0 to 5","examples":["f(2, 0, 4) = -2","f(3, 3, 1) = -42","f(4, 4, 3) = -95","f(4, 5, 2) = -176","f(5, 0, 3) = -2","f(5, 5, 2) = -176","f(5, 5, 3) = -179"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>, <int>) or TRACE input=(<int>, <int>, <int>) line=<n>; at most 4 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":4,"used":0,"remaining":4},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":7,"python_runs":0},"hidden_truth":{"bug_line":6,"bug_kind":"variable","shown_line":"score = carry - b + score","intended_expression":"score - b + score"},"gold_commit":"COMMIT line=6 fix=score - b + score","gold_commit_verified":true,"truth_sha256":"669517f3a899508183ac17b43af2926db96b6e44d9f3f4cbed660f1196f30d6b"}
{"family":"trace_debugging","level":2,"split":"dev","seed":900007,"instance_hash":"47bfdacbcf0d399fb7ba9ff0aafb5cb301fe525d3366343d810127c420215688","observation":{"program":["def f(a, r, m):"," 1|  run = a"," 2|  step = 0"," 3|  high = run * 2"," 4|  for j from 2 to a + 1:"," 5|      if run % 2 == 0:"," 6|          high = high + j * r"," 7|      else:"," 8|          run = run + step * 3"," 9|      step = step + a * 2","10|  run = high + 3","11|  high = high - r * run","12|  return high - step"],"language":"integers only. 'v = E' assigns. 'for v from A to B:' runs the indented body for v = A, A+1, .., B (not at all if A > B; v exists only in the loop). 'if C:' runs its body if C holds, else the 'else:' body if any. 'return E' returns E. + - * and % (remainder 0..k-1, k > 0); * and % before + and -, left to right; ( ) group. C is E op E, op one of < <= > >= == !=.","input_domain":"f(a, r, m) with every argument an integer from 0 to 5","examples":["f(0, 3, 1) = -13","f(1, 0, 1) = 0","f(1, 4, 4) = -62","f(2, 3, 2) = -55","f(2, 5, 1) = -31","f(3, 4, 3) = -48","f(5, 4, 1) = -68"],"bug":"exactly one line differs from the intended program, by one edit: an operator swapped within its kind (+ - *, or < <= > >= == !=), an integer constant off by one, or a variable read replaced by another variable defined at that line; targets, loop variables and structure are as intended; some examples may already pass","tools":"RUN: the shown program's output. TRACE: the variables defined at line n just before each run of line n (first 12 runs; an else line: on entering the else branch)","objective":{"kind":"repair_program","instruction":"find the buggy line and its corrected expression","report":["line","fix"],"success_rule":"the COMMIT succeeds only if the program with that line's expression replaced by the fix returns the intended value for every input of the domain; a malformed action, an input outside the domain, a variable not defined at that line or an action beyond the budget ends the episode at zero"},"action_grammar":"RUN input=(<int>, <int>, <int>) or TRACE input=(<int>, <int>, <int>) line=<n>; at most 4 in total. Then: COMMIT line=<n> fix=<that line's expression: right side of an assignment or return, condition of an if, 'A to B' of a for>, e.g. COMMIT line=4 fix=acc + 2 * i","experiment_budget":{"maximum":4,"used":0,"remaining":4},"history":[],"status":"active","world":"trace_debugging_bench"},"turn_limits":{"lab_action_turns":7,"python_runs":0},"hidden_truth":{"bug_line":1,"bug_kind":"variable","shown_line":"run = a","intended_expression":"m"},"gold_commit":"COMMIT line=1 fix=m","gold_commit_verified":true,"truth_sha256":"503d64f38e147e5ec48c0f16290abd38d34b883e6e035f02253481eedc5bbcdd"}
