NLP103 [26-1] 김효민 - GQA: Training Generalized Multi-Query Transformer Models fromMulti-Head Checkpoints [Paper]GQA : https://arxiv.org/abs/2305.13245 GQA: Training Generalized Multi-Query Transformer Models from Multi-Head CheckpointsMulti-query attention (MQA), which only uses a single key-value head, drastically speeds up decoder inference. However, MQA can lead to quality degradation, and moreover it may not be desirable to train a separate model just for faster inference. We (1) prarxiv.org MQ.. 2026. 2. 28. [2026-1] 박승원 - RoFormer: Enhanced Transformer with Rotary Position Embedding 논문 정보게재 학술지: Neurocomputing, Volume 568, 1 February 2024, 127063저자: (Zhuiyi Technology)Jainlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, Yunfeng Liuhttps://arxiv.org/abs/2104.09864 RoFormer: Enhanced Transformer with Rotary Position EmbeddingPosition encoding recently has shown effective in the transformer architecture. It enables valuable supervision for dependency modeling between elem.. 2026. 2. 27. [2026-1] 김지원 - LoBERT: Generative AI Foundation Model for Limit Order Book Messages 논문 소개제목: LoBERT: Generative AI Foundation Model for Limit Order Book Messages(2025)논문 링크: https://arxiv.org/pdf/2511.12563논문 정보: LOB Level 3 시퀀스 데이터를 양자화 및 트랜스포머 모듈을 통해 모델링에 성공함 초록금융 지정가 주문장(LOB)의 역학을 메시지 수준에서 모델링하는 것은 불규칙적인 이벤트 발생 시점, 급격한 시장 상황(regime) 변화, 그리고 고빈도 거래자들이 가시적인 주문 흐름에 보이는 반응 때문에 어렵다.기존의 LOB 모델들은 복잡한 데이터 표현 방식을 요구하고 원래의 작업 외에는 적응성이 부족했다. 이러한 문제점을 해결하기 위해, 본 논문에서는 다운스트림 미세 조정에 적합한 범.. 2026. 2. 21. [2026-1] 김정운, 김효민 - CritPT (Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics Research Benchmark) [CritPt] Tech Report & Example Challenge & Dataset Link CritPt - Physics Benchmark critpt.com Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics Research BenchmarkWhile large language models (LLMs) with reasoning capabilities are progressing rapidly on high-school math competitions and coding, can they reason effectively through complex, open-ended challenges found in front.. 2026. 2. 19. 이전 1 2 3 4 ··· 26 다음
