03 - Land use, water, soil, N and SOC review of cleaned v0.7.0

Reviewer role: sustainable land-use and agro-environmental modeller (adversarial review) Source reviewed (read-only): CIAT/cleaned @ 800d53a (v0.7.0) Scope: land_requirement.R, land_productivity.R, biomass_calculation.R, water_requirement.R, soil_health.R, soc.R, nitrogen_balance.R, compare_scenario.R, differences.R, merge_outputs.R, inst/extdata/*.json, vignettes, README.

Verdict

The land, water, erosion, N-balance and SOC modules should not be used for quantitative ex-ante claims yet. The land footprint silently drops feeds (any Inf/NaN area becomes 0), so up to 74% of the ration's dry matter got no land in a test run. It is also inflated by up to about 70% because feed demand is set to the larger of the energy-based and protein-based DMI. The N balance adds per-hectare terms to farm totals, has a scalar ifelse bug that throws away grazing-manure N, silently ignores fertiliser types missing from the fertiliser table, and calls outputs/inputs including losses "NUE". Water is reported in mm·ha labelled as m³ (10x too low), with no green/blue split and no season length. SOC applies IPCC factors to a measured stock of arbitrary depth, so any baseline shows a permanent loss, and tree-SOC is always zero. There is no uncertainty treatment. The scenario % changes flip sign for negative baselines and report 0% for undefined ones.

How numbers were checked

Findings table

id severity verdict file:line issue
LU-01 Critical CONFIRMED land_requirement.R:99-102, 120-121 Division by zero (removal=0, yield=0, source_type not Main/Residue) gives Inf/NaN, which is then set to 0 ha. Feeds silently get no land (74% of DM in QT).
LU-02 High CONFIRMED energy_requirement.R:174; land_requirement.R:93 Land is driven by dmi_s = max(DMI_energy, DMI_CP). CP-limited diets inflate DM demand and land by 27-69% in the examples.
LU-03 Medium PLAUSIBLE land_requirement.R:103-107 Residue land uses undocumented mass allocation. Main items take 100% of crop land even when their residue is also fed, so there is a double-count risk. No economic allocation.
LU-04 Medium CONFIRMED land_requirement.R:108-110; water_requirement.R:53-55; nitrogen_balance.R:269-298; merge_outputs.R:602-604 Off-farm/imported status is inferred from the substrings "OFR"/"OFC"/"IP" in feed names. Off-farm land is then scored with on-farm soil and climate.
LU-05 Low PLAUSIBLE land_requirement.R:100-102 Area is cut by intercrop*intercrop_fraction with no documented meaning.
LU-06 Medium CONFIRMED differences.R:211-214; compare_scenario.R:328-342; differences.R:195 vs merge_outputs.R:530 "ha/kg" indicators are really ha/t (x1000). Milk, meat and protein each get 100% of the land (no co-product allocation). AME uses 2100 vs 2500 kcal in different places.
N-01 Critical CONFIRMED nitrogen_balance.R:249-256 Leaching out3 is a per-ha rate that is never multiplied by area. It also adds total organic N (in2) into a per-ha pool, and is then summed with farm-total terms in nout.
N-02 High PLAUSIBLE nitrogen_balance.R:249 Leaching coefficient written as 0.021*(P-3.9). The De Willigen/NUTMON form is 0.021*P - 3.9, so leaching is about 3.8 percentage points too high (+22% at 1000 mm, +84% at 400 mm).
N-03 High CONFIRMED nitrogen_balance.R:234-238 ifelse(denom > 0, vector, 0) returns only the first element. Grazing-manure N is replaced by feed 1's value for every feed (126.6 kg N lost in QT).
N-04 High CONFIRMED nitrogen_balance.R:80-121; example_input.json fertilizer A fertiliser missing from para$fertilizer, or with no percentage_n, silently gives 0 N. QT's 400 kg urea/ha becomes 0 N, while GHG manufacturing emissions still count it.
N-05 High CONFIRMED nitrogen_balance.R:148-155 All collected-manure N and purchased organic N is given to every feed x fraction_as_fertilizer, with no normalisation. N is multiplied by the number of feeds (200% in QT).
N-06 High CONFIRMED nitrogen_balance.R:257, 285-294 "NUE" = nout/nin, where nout includes leaching, gaseous and erosion losses. The mining (>0.9) and leaching (<0.5) classes are meaningless (EX NUE = 24 and 54).
N-07 Medium CONFIRMED nitrogen_balance.R:164 vs 184 soil_n is used on two bases 10x apart. The soil N stock (and so mineralisation) is 10x low if soil_n is in g/kg, which is the basis erosion N (out5) assumes.
N-08 Medium CONFIRMED nitrogen_balance.R:174, 253-254 Non-symbiotic fixation 2+(P-1350)*0.005 is negative when P < 950 mm. Gaseous loss can go negative at low clay and low rain. Neither is clamped.
N-09 Medium PLAUSIBLE nitrogen_balance.R:143-148; ghg_emission.R:373-375 The N balance uses excreted N with no storage losses (not consistent with the GHG manure-N chain). The GHG module uses fertiliser N as off-farm residue N.
SE-01 High CONFIRMED soil_health.R:35-58, 73; nitrogen_balance.R:179-182 LS is NA when slope_desc is empty/unknown or slope_length is NA. The NA is later dropped with na.rm, so erosion and erosion-N become 0. EX: all slope_desc = "" gives zero erosion.
SE-02 High PLAUSIBLE soil_health.R:26-27 R = 0.55*(P/rain_length) - 4.7 has no source. It gives about 98-160, roughly 5-10x below common SSA erosivity relations. The units of K are unstated.
SE-03 Medium PLAUSIBLE soil_health.R:34-58; example inputs slope_p_factor The LS lookup has no source, units or length basis, and uses very wide slope classes. P = 0.11 (terracing level) is the de facto default.
SE-04 High CONFIRMED merge_outputs.R:620, 649-665 erosion_t_soil_year is built in the order total, OFR, OFC, IP, farm but labelled total, on-farm, OFR, OFC, IP. Rows are shifted, and per-ha values use the wrong area.
SE-05 Medium CONFIRMED merge_outputs.R:628-634; differences.R:231 balance_N_kg_N_ha sums per-ha values across feeds. The headline N balance uses the whole-crop balance in one output and the feed-only balance in another.
W-01 High CONFIRMED water_requirement.R:48-49, 57, 74-81; differences.R:352-376 ET in mm x area in ha is reported as "m3" and "m3/ha" (1 mm·ha = 10 m³), so values are 10x too low.
W-02 High CONFIRMED water_requirement.R:33-49 Annual ET0 x mean Kc is applied over 365 days for every crop. No growing season, effective rainfall, irrigation, green/blue split or drinking water. The wfp_green/blue/grey and livestock water_requirement inputs are unused.
W-03 Medium CONFIRMED water_requirement.R:49-51, 74 Residue water is allocated twice (already-allocated area_feed x feed share again). total_water_use does not equal the sum of feed_water_use (QT test: 307 vs 1460 mm/ha).
W-04 Medium CONFIRMED water_requirement.R:56 kc_water_use_on_farm subtracts roughages and IP but not OFC concentrates.
W-05 Low CONFIRMED water_requirement.R:46 Kc is the plain mean of the three stages, not a stage-duration-weighted mean (FAO-56).
S-01 High CONFIRMED soc.R:234; biomass_calculation.R:141-144 biomass[["trees_non_feed_biomass"]] is NULL (biomass is a plain data frame), so tree-SOC is always 0.
S-02 High CONFIRMED soc.R:103-133 IPCC F factors are applied to the measured field SOC as if it were SOCref. Long-term, steady-state cropland therefore shows a permanent loss (QT: -17 t C/ha/yr).
S-03 High CONFIRMED soc.R:73-74; input-contract.Rmd:53 soil_depth is implicitly in metres, with no 0-30 cm basis and no validation. EX (depth 2) gives 832 t C/ha; QT (BD = 6) gives 1440 t C/ha.
S-04 High CONFIRMED stock_change_parameters.json grassland.management."Medium": 1.5 This management factor is not in IPCC. It gives +50% SOC over 20 years (about +1 t C/ha/yr on the fixed 40 t C).
S-05 Medium CONFIRMED soc.R:106-107, 141-142, 175-176 "Cropland" area includes off-farm and imported concentrate land at farm SOC. Grassland SOC is fixed at 40 t C and ignores field_soc. Tree-legume land gets no SOC.
S-06 Low CONFIRMED output-contract.Rmd:258-262 vs soc.R:245-248 The docs promise field_soc and per-land-use changes; the function returns only 2 totals.
C-01 High CONFIRMED compare_scenario.R:49-150 (all % blocks) % change = (S-B)/B. It flips sign for negative baselines (N balance, SOC) and gives 0% when B = 0.
C-02 Medium CONFIRMED differences.R:39-40; water_requirement.R:76-81; merge_outputs.R:703-704 Zero denominators give 0 intensities (for example 0 m³ or 0 kg CO2e per kg meat when there is no meat) instead of NA.
C-03 High CONFIRMED all modules (absence) Everything is deterministic. There is no uncertainty, sensitivity or range output, although results hinge on unobserved inputs (yield, HI, removal, Kc, R, LS, P, soil N).
D-01 High CONFIRMED README.md "Standard Model Run"; inst/extdata/example_input.json The shipped example and README workflow fail (missing crop_name, dry_yield, residue_dry_yield, percentage_n, fat_milkcontent, cp_lys_*, n_manure_content). The derived yields are computed outside the package.
D-02 Medium CONFIRMED input-contract.Rmd:42-53, 146; example_input.json Units are given only as the "package basis". dm_content is a fraction (0.3) in EX but a percent (89) in QT and in the README. No range checks (BD = 6, depth = 2).

Counts: Critical 2, High 18, Medium 13, Low 3 (36 in total).


Detailed findings

Land requirement

LU-01 - Silent zero land for feeds (Critical, CONFIRMED)

LU-02 - Land driven by the larger of energy- and CP-based DMI (High, CONFIRMED mechanism)

LU-03 - Allocation of crop land between food and residue feed (Medium, PLAUSIBLE)

LU-04 - Feed-source classification by name substring (Medium, CONFIRMED)

LU-05 - Intercropping reduction (Low, PLAUSIBLE)

LU-06 - Land intensity metrics (Medium, CONFIRMED)

Nitrogen balance

N-01 - Leaching term mixes per-ha and total units (Critical, CONFIRMED)

N-02 - Leaching regression mis-parenthesised (High, PLAUSIBLE)

N-03 - Scalar ifelse discards grazing-manure N (High, CONFIRMED)

N-04 - Missing fertiliser types or N content silently give 0 N (High, CONFIRMED)

N-05 - Manure and purchased organic N multiplied across feeds (High, CONFIRMED)

N-06 - "NUE" is not NUE (High, CONFIRMED)

N-07 - Two bases for soil_n (Medium, CONFIRMED)

N-08 - Negative inputs and losses (Medium, CONFIRMED)

N-09 - Cross-module N consistency (Medium, PLAUSIBLE)

Soil erosion

SE-01 - Erosion silently zero when slope inputs are missing (High, CONFIRMED)

SE-02 - Rainfall erosivity formula (High, PLAUSIBLE)

SE-03 - LS table and P default (Medium, PLAUSIBLE)

SE-04 - Erosion summary rows shifted (High, CONFIRMED)

SE-05 - Per-ha N summary summed across feeds (Medium, CONFIRMED)

Water

W-01 - mm·ha reported as m³ (High, CONFIRMED)

W-02 - ET method incomplete (High, CONFIRMED)

W-03 - Double allocation for residues (Medium, CONFIRMED)

W-04 - OFC included in on-farm water (Medium, CONFIRMED)

W-05 - Plain-mean Kc (Low, CONFIRMED)

Soil organic carbon and biomass

S-01 - Tree SOC always zero (High, CONFIRMED)

S-02 - Permanent SOC loss for steady-state land (High, CONFIRMED)

S-03 - Depth and units (High, CONFIRMED)

S-04 - Non-IPCC grassland factor (High, CONFIRMED)

S-05 - Land categories (Medium, CONFIRMED)

S-06 - Doc mismatch (Low, CONFIRMED)

Scenario comparison and uncertainty

C-01 - % change arithmetic (High, CONFIRMED)

C-02 - Zero-denominator intensities (Medium, CONFIRMED)

C-03 - No uncertainty or sensitivity (High, CONFIRMED by absence)

Documentation

D-01 - Shipped examples do not run (High, CONFIRMED)

D-02 - Units unstated or inconsistent (Medium, CONFIRMED)


Done well

Open questions for authors

  1. What are the intended units of soil_n, soil_c, soil_depth, soil_k_value, rain_length, slope_length and et, and what is the source of the R formula and the LS table?
  2. Where (app or batch) are dry_yield, residue_dry_yield and crop_name derived, and can that code be moved into the package and tested?
  3. Is mass allocation of crop land to residues a deliberate choice? Would you accept economic allocation (FAO LEAP) as an option?
  4. Should land and feed demand follow energy-limited intake rather than max(energy, CP)?
  5. Is SOC change meant to be baseline-to-scenario (management change) or native-to-current? Is the grassland "Medium" = 1.5 factor intentional?
  6. What should a zero removal fraction on a fed item mean? And how should "Purchased" feeds get a land and water footprint?
  7. Is any validation dataset (for example NUTMON farm studies or measured erosion plots in East Africa) available to benchmark the N balance and erosion magnitudes?
  8. Should milk and meat footprints be allocated (IDF 2015 or economic) rather than each carrying 100% of the impacts?

Comments

Comments are GitHub Discussions on peetmate/cleaned-review; sign in with GitHub to post.