Tasks that require an agent to accomplish a complex sequence of events in the environment are common in practice. However, reward functions that capture such tasks do not satisfy the Markov property, which is necessary to ensure the correctness and convergence properties of classical reinforcement learning algorithms. This thesis introduces the concept of reward machines and a related reinforcement learning algorithm: Q-learning for Reward Machines (QRM). Reward machines are a type of finite automaton that can capture reward functions that do not satisfy the Markov property. They can be understood as a form of finite memory in the Markov decision process. Unfortunately, writing such formalisms is a difficult manual task, and there is a need to simplify this process. Some methods approach this by learning automata jointly with the policy, but they assume a predefined reward signal that conforms to a finite-state structure. This thesis explores a different approach: using new technologies from the field of natural language processing, aiming to facilitate the process of defining reward signals. For this purpose, a language model based on the well-known GPT-2 model was fine-tuned, and the process of data augmentation that was employed to create the training dataset is described in detail. The final result enables the following steps. 1. The user describes the task in English, for example, “patrol the forest and the factory, but avoid traps”. 2. The language model is invoked to translate the user’s description into a reward machine formalism. 3. QRM and the reward machine from Step 2 are used to train the agent to perform the task described by the user in Step 1. 4. The results are visually demonstrated to the user in a small number of episodes. The thesis presents and analyzes the results of training the language model, as well as the results of the QRM algorithm paired with the reward machines discovered using the model. Additionally, a graphical user interface has been developed to demonstrate the usage of the language model and the training results of the agent.