How popular neural networks cope with various tasks in social sciences and humanities (an empirical research)
E. V. Biricheva
Institute for Philosophy and Law Ural Branch Russian Academy of Sciences, Yekaterinburg, Russia
Keywords: education, neural network, NN, large language model, LLM, DeepSeek, Yandex Alice, social and humanitarian disciplines, pedagogical problems, text generating, artificial intelligence
Abstract
Introduction. The use of neural networks by students to do written works seems to be one of the new challenges for the entire educational system. The pedagogical problem is, first of all, that in these cases the competencies prescribed by the standards are not formed. These tools may generate false answers that, in addition, may violate ethical norms or requirements of logic, objectivity, and impartiality. Thus, the purpose of the work was to study the capabilities of popular neural networks in solving various types of tasks in social and humanitarian disciplines, as well as to draw up recommendations for teachers on the transformation of the pedagogical process, taking into account the new realities. Methodology. We distinguished 10 types of tasks typical for disciplines of the social and humanitarian cycle, and loaded them into the two most popular models, DeepSeek and Alice (Yandex). The responses of neural networks to 500 queries were evaluated qualitatively (by depth, content, logic, etc.) and were rated quantitatively (in points and percent). Then we compared these responses with each other and with students’ grades on similar tasks. Discussion. Analysis of empirical data showed that neural networks, on average, cope well with such tasks as giving a definition, listing features or positions, arguing a point of view on a problem. Some difficulties arose with search tasks (indicating personalities, determining the affiliation of a quote, etc.) and commenting tasks (interpreting a quote taking into account its context in the original source). It may be due to the peculiarities of machine learning and data unpacking, as well as translations and the availability of Russian-language sources for a Chinese neural network. The examined neural networks write complex tasks (essays, projects) rather superficially, with low heuristics, without referring to literature, etc., if skillful prompt engineering was not used. Conclusion. Based on the results, recommendations were formulated for compiling tasks taking into account the strengths and weaknesses of popular neural networks. Concrete advices were also given on the diversity of types of work and the definition of text generating
|
