The automation of fossil pollen analysis promises many advantages in handling large numbers of samples with less resource allocation. However, automation is often obstructed by the high abundance of organic and minerogenic non-pollen debris in fossil pollen samples. We used a Convolutional Neural Network-based approach to detect pollen-like objects in digital images of prepared microscopic slides for fossil pollen analysis and subsequently classified them into nine pollen classes and the marker spore Lycopodium. We trained the object detection and the classification model independently with a newly developed dataset of annotated images of fossil pollen grains. The object detection model achieved average recall rates of 89.8 % and 75.5 % for pollen classes and Lycopodium, respectively. The classification model correctly categorizes fossil pollen images with >95 % accuracy. We applied the assembled pipeline to Late Glacial pollen samples using class-dependent thresholds to discriminate true pollen from non-pollen objects and compared automated count data for nine pollen types with manual pollen counts. For the selected pollen types, our results demonstrate the feasibility to replicate major fossil pollen changes with automated counts, even when the automated pipeline was applied to pollen samples from a different site than used to train the models. High correlations (r = 0.97) between the first two axes of Principal Component Analyses (PCA) calculated based on automated and manual counts and high correlation (r = 0.93) indicated by a Procrustes rotation analysis of the PCA results demonstrate that the two procedures reconstructed similar pollen patterns. While our automated approach is not yet able to achieve the taxonomic resolution of manual counts by expert analysts and is limited to selected pollen types, it provides a “proof of principle” that automated analyses can be applied to complex fossil pollen samples and to develop downcore stratigraphies. Automated analyses may with time lead to reliable pollen records. For instance, our pipeline can be further improved by adding more pollen classes, increasing the dataset of annotated images of fossil pollen grains, expanding the training data to rare pollen types, refining taxonomic resolution (e.g., separation of Betula nana-type or Pinus-types), and incorporating more challenging pollen types (e.g., Juniperus), to expand its application beyond reconstructing temporal changes in a few selected pollen types.