Gold nanoparticles (AuNPs) have been extensively explored in biomedical applications, for example as drug carriers, contrast agents, or therapeutics. However, AuNP can exhibit cytotoxic profile, when the size is below 2 nm (ultrasmall AuNP; usAuNP) and when the stabilizing ligands allow for access to the gold surface either for the direct interaction with biomolecules or for catalytic activity of the unshielded gold surface. Furthermore, usAuNP exhibits significantly different biodistribution and enhanced circulation times compared to larger AuNP. This review gives an overview about the synthesis and the physico-chemical properties of usAuNP and, thereby, focusses on 1.4 nm sized AuNP, which are derived from the compound Au55(PPh3)12Cl6 and which are the most intensively studied usAuNP in the field. This part is followed by a summary of the toxic properties of usAuNP, which include in vitro cytotoxicity tests on different cell lines, electrophysiological tests following FDA guidelines as well as studies on antibacterial effects. Finally, the biodistribution and pharmacokinetics of ultrasmall AuNP are discussed and compared to the properties of more biocompatible, larger AuNP.