There is substantial interest in assessing Human Reliability Analysis (HRA) methods and ultimately in validating the approaches and models underlying these methods. A significant, initial step in this direction is an international evaluation study of HRA methods, based on comparing observed crew performance in the Halden Reactor Project's HAMMLAB (HAlden huMan-Machine LABoratory) simulator with the outcomes predicted in HRA analyses. The aim of the study is to develop an empirically-based understanding of the performance, strengths, and weaknesses of the methods. This paper presents the methodology for this assessment study and selected results and insights obtained in an initial "pilot" phase. It covers the design of the scenarios and experimental data collection and analysis, the predictive HRA analyses, and the approaches for the comparison of the predicted and observed outcomes. The comparison of HRA method results with a set of empirically-based reference performance data is a prominent feature of this effort. Furthermore, the paper summarizes the strengths and weaknesses of the overall design of the study and insights for future studies.