A suite of software tools has been developed for dose estimation (BAT, WinFRAT) and prediction of acute health effects (WinFRAT, H-Module) using clinical symptoms and/or changes in blood cell counts. We constructed a database of 191 ARS cases using the METREPOL (n = 167) and the SEARCH-database (n = 24). The cases ranged from unexposed (RC0), to mild (RC1), moderate (RC2), severe (RC3), and lethal ARS (RC4). From 2015-2019, radiobiology students and participants of two NATO meetings predicted clinical outcomes (RC, H-ARS, and hospitalization) based on clinical symptoms. We evaluated the prediction outcomes using the same input datasets with a total of 32 teams and 94 participants. We found that: (1) unexposed (RC0) and mildly exposed individuals (RC1) could not be discriminated; (2) the severity of RC2 and RC3 were systematically overestimated, but almost all lethal cases (RC4) were correctly predicted; (3) introducing a prior education component for non-physicians significantly increased the correct predictions of RC, ARS, and hospitalization by around 10% (p<0.005) with a threefold reduction in variance and a halving of the evaluation time per case; (4) correct outcome prediction was independent of the software tools used; and (5) comparing the dose estimates generated by the teams with H-ARS severity reflected known limitations of dose alone as a surrogate for H-ARS severity. We found inexperienced personnel can use software tools to make accurate diagnostic and treatment recommendations with up to 98% accuracy. Educational training improved the quality of decision making and enabled participants lacking a medical background to perform comparably to experts.