With the development of smart education, gaining insights into students' understanding during the learning process is crucial in teaching. However, traditional knowledge tracking methods face challenges in capturing the intricate relationships between problems and knowledge points, as well as students' temporal learning changes. Therefore, we design a knowledge tracking model based on a graph temporal fusion network. Firstly, we construct the structure of the question and knowledge skill graph. Then, we design a knowledge graph encoder layer to capture the complex relationships between questions and knowledge skills. Next, we apply a sequential information extraction layer to dynamically model the outputs of each layer in the upper network over time, capturing students' knowledge changes at different time steps. Finally, we use a dynamic attention aggregation network to learn node information at different levels and time sequences. Experimental results on three datasets demonstrate the effectiveness of our method.