Cell-penetrating peptides (CPPs) are prominent delivery vehicles to confer cellular entry of (bio-) macromolecules. Internalization efficiency and uptake mechanism depend, next to the type of CPP and cargo, also on cell type. Direct penetration of the plasma membrane is the preferred route of entry as this circumvents endolysosomal sequestration. However, the molecular parameters underlying this import mechanism are still poorly defined. Here, we make use of the frequently used HeLa and HEK cell lines to address the role of lipid composition and membrane potential. In HeLa cells, at low concentrations, the CPP nona-arginine (R9) enters cells by endocytosis. Direct membrane penetration occurs only at high peptide concentrations through a mechanism involving activation of sphingomyelinase which converts sphingomyelin into ceramide. In HEK cells, by comparison, R9 enters the cytoplasm through direct membrane permeation already at low concentrations. This direct permeation is strongly reduced at room temperature and upon cholesterol depletion, indicating a complex dependence on membrane fluidity and microdomain organisation. Lipidomic analyses show that in comparison to HeLa cells HEK cells have an endogenously low sphingomyelin content. Interestingly, direct permeation in HEK cells and also in HeLa cells treated with exogenous sphingomyelinase is independent of membrane potential. Membrane potential is only required for induction of sphingomyelinase-dependent uptake which is then associated with a strong hyperpolarization of membrane potential as shown by whole-cell patch clamp recordings. Next to providing new insights into the interplay of membrane composition and direct permeation, these results also refute the long-standing paradigm that transmembrane potential is a driving force for CPP uptake.