Molecular techniques using DNA retrieved from community or environmental samples, in particular environmental DNA (eDNA), are becoming increasingly popular for detecting individual species, assessing biodiversity, and quantifying ecological indices. More recently, eDNA has also been proposed as a template for population genetics, and several studies have already tested the feasibility of this approach, mostly looking at vertebrate species. Their results along with general opportunities offered by these types of "community-based" samples, such as the possibility to target multiple species at the same time, have generated great enthusiasm and expectations for using eDNA in population genetics. However, not every aspect of population genetics can be addressed by eDNA-based data and some inherent limitations may challenge its conclusions. Here, we firstly review the state of current knowledge of DNA retrieved from environmental and community samples for population genetics. Then, focusing on eDNA, we summarize the opportunities but also detail four main limitations of its use for population-level inferences, namely, (1) the difficulty to retrieve a species-specific dataset, (2) the potential lack of correlation between observed and true allelic frequencies, (3) the loss of individual information in multi-locus genotyping and linkage between loci, and (4) the uncertainty about the individuals contributing to the sampled DNA pool (e.g., number, life-stage, or sex). Some of these limitations might be overcome with the development of new technologies or models that account for the specificities of eDNA. Others, however, are inherent, and their effect on the inferences must be thoroughly evaluated. The possibility of gaining insights into genetic diversity and population structure from DNA retrieved from community and environmental samples is appealing for scientists, conservation managers, and other practitioners. Yet, to avoid false expectations and incorrect inferences, it is imperative that these limitations are known and considered alongside the opportunities and advantages.