Enhanced matrix inference with Seq2seq models via diagonal sorting

Abstract

AbstractThe effectiveness of sequence-to-sequence (seq2seq) models in natural language processing has been well-established over time, and recent studies have extended their utility by treating mathematical computing tasks as instances of machine translation and achieving remarkable results. However, our exploratory experiments have revealed that the seq2seq model, when employing a generic sorting strategy, is incapable of inferring on matrices of unseen rank, resulting in suboptimal performance. This paper aims to address this limitation by focusing on the matrix-to-sequence process and proposing a novel diagonal-based sorting. The method constructs a stable ordering structure of elements for the shared leading principal submatrix sections in matrices with varying ranks. We conduct experiments involving maximal independent sets and Sudoku laws, comparing seq2seq models utilizing different sorting methods. Our findings demonstrate the advantages of the proposed diagonal-based sorting in inference, particularly when dealing with matrices of unseen ranks. By introducing and advocating for this method, we enhance the suitability of seq2seq models for investigating the laws of matrix inclusion and exploring their potential in solving matrix-related tasks.