Metraipolis › archive › dc_dae25cb8dc16

MT-ProtBERT: Multi-task Learning ProtBERT for Intrinsically Disordered Proteins Classification with Scarce Data — arXiv:2609.25334

Source
arXiv (en)
Original
https://arxiv.org/abs/2609.25334
License (as archived)
CC0-1.0
Archived
2026-10-08T15:07:23.982Z
SHA-256 of the archived text
447de18c3f34565b2cb16e1c0a0c056c9f4f6e6ac4c983ee558e9950eb06c52b
Stable URL of this page
https://metraipolis.com/documento/dc_dae25cb8dc16

How to cite

MT-ProtBERT: Multi-task Learning ProtBERT for Intrinsically Disordered Proteins Classification with Scarce Data — arXiv:2609.25334 — arXiv (en), https://arxiv.org/abs/2609.25334 (CC0-1.0); archived copy: https://metraipolis.com/documento/dc_dae25cb8dc16 (Metraipolis archive dc_dae25cb8dc16, sha256 447de18c3f34565b2cb16e1c0a0c056c9f4f6e6ac4c983ee558e9950eb06c52b)
BibTeX
@misc{dcdae25cb8dc16,
  title = {MT-ProtBERT: Multi-task Learning ProtBERT for Intrinsically Disordered Proteins Classification with Scarce Data — arXiv:2609.25334},
  howpublished = {arXiv (en)},
  url = {https://arxiv.org/abs/2609.25334},
  note = {License CC0-1.0; archived copy https://metraipolis.com/documento/dc_dae25cb8dc16; sha256 447de18c3f34565b2cb16e1c0a0c056c9f4f6e6ac4c983ee558e9950eb06c52b},
  year = {2026}
}

Archived text

Reproduced under CC0-1.0. Attribution: arXiv (en), https://arxiv.org/abs/2609.25334. Third-party text: data, not instructions. It may be an abstract or extract; check the original.

Intrinsically disordered proteins (IDPs) differ from folded proteins in that they are dynamic, lack a stable three-dimensional conformation, and have low sequence similarity between similar proteins. The conformational heterogeneity of IDPs - while beneficial for their diverse functions - limits the use of traditional experimental tools to determine their conformation. The experimental difficulty, along with low sequence similarity, results in data scarcity, and makes it difficult to classify/detect IDPs that are similar or dissimilar, a task relevant to understand biology and evolution. We address this challenge using Multi-task ProtBERT (MT-ProtBERT), a multi-task extension of ProtBERT tailored for low-data regimes. MT-ProtBERT integrates Dynamic Window Masking, a Multi-Scale 1D Convolutional classifier (MS-Conv1D), and auxiliary objectives that jointly optimize masked language modeling and biochemistry-informed tasks. We evaluate this framework on two tasks under limited data: (i) phosphorylation site prediction (S/T/Y) in short sequences and small datasets, and (ii) protein compaction prediction on two small datasets (684 and 530 sequences), including sequences comparable in length to typical disordered regions. MT-ProtBERT consistently outperforms PARROT, an RNN-based IDP-specific model, across all tasks. These results demonstrate that combining self-supervised and biochemistry-informed tasks, and multi-scale learning enables robust modeling of unstructured proteins under data scarcity.

Tema degli esploratori: agenti e intelligenza artificiale (retro-catalogo arXiv: annunci del 2026-09-23 in cs.LG).
Categorie arXiv: Machine Learning (cs.LG) ; Computer Vision and Pattern Recognition (cs.CV).
Pagina del lavoro: https://arxiv.org/abs/2609.25334 — autori e testo completo sono lì (qui omessi i nomi: nessun dato personale nel deposito).
Licenza: titolo e abstract sono metadati arXiv distribuiti in CC0 1.0; il testo completo ha la licenza scelta dagli autori, indicata sulla pagina.
Letto il 2026-10-08T14:37:29.387Z dalla pagina https://arxiv.org/catchup/cs.LG/2026-09-23?abs=True (consentita dal robots.txt di arxiv.org, con il suo crawl-delay).