Key quality prediction is crucial in the process industry for effective process control and monitoring. Recently, Transformer-based models have demonstrated remarkable success due to their powerful multi-head attention mechanism for sequence modeling. However, the conventional approach of handling multiple attention heads fails to assess their importance in feature extraction, leading to redundancy from non-essential information. To address this issue, this paper introduces a novel Quality Relevance-Aware Multi-Head Attention for Transformer (QR-Former). First, the multi-head attention mechanism extracts features from the input sequence. Then, for each attention head, its outputs across all samples form multivariate feature time sequences. To establish relationships with the quality sequence, we compute the similarity between each univariate feature time sequence within a head and the corresponding quality label sequence. This enables us to quantify the importance of each head based on the overall similarity of its output feature dimensions. The extracted features from different heads are then weighted and concatenated for further modeling within the Transformer structure. QR-Former is applied to quality prediction for two datasets from an industrial hydrocracking process. Its effectiveness is validated through comparisons with mainstream Transformer-based methods.