OpenAlex is an online repository using an automated classification system. Our findings suggest that the system has significant hierarchical and semantic shortcomings identified through a mixed-method research approach. This approach utilized qualitative and quantitative research methods to provide a holistic evaluation of the system, gauging its effectiveness in indexing papers. Moreover, we provide key insights into how improvements can be made to render OpenAlex an effective and usable classification system.