Synergistic drug combinations are commonly sought to overcome monotherapy resistance in cancer treatment. To identify such combinations, high-throughput cancer cell line combination screens are performed; and synergy is quantified using competing models based on fundamentally different assumptions. Here, we compare the behaviour of four synergy models, namely Loewe additivity, Bliss independence, highest single agent and zero interaction potency, using the Merck oncology combination screen. We evaluate agreements and disagreements between the models and investigate putative artefacts of each model's assumptions. Despite at least moderate concordance between scores (Pearson's r >0.32, Spearman's rho > 0.34), multiple instances of strong disagreement were observed. Those disagreements are driven by, among others, large differences in tested concentrations, maximum response values and median effective concentrations.