Gene-based rare variant association tests (RVATs) are essential for uncovering disease mechanisms and identifying candidate drug targets, yet existing frameworks lack flexibility in integrating multiple variant annotations. Here, we introduce BayesRVAT, a Bayesian framework for RVAT which models variant effects using priors informed by multiple annotations. We show that BayesRVAT outperforms state-of-the-art burden test strategies in both simulations and an analysis of 12 blood traits from the UK Biobank.