The Open Source LLM Revolution: RedPajama-INCITE Models Leading the Charge!
- [ð] ã«ãŒã¯ã»ã¢ãã³ãã³æ°ã«ãããããªã§ã¯ãæ°ãã«ãªãªãŒã¹ããããªãŒãã³ãœãŒã¹ã®ã¢ãã«ãRed Pajamaãã«ã€ããŠèª¬æããŠããŸãããã®ã¢ãã«ã¯ãåçšå©çšå¯èœãªå®å šã«ãªãŒãã³ãœãŒã¹ã®ããŒã¿ã»ãããäœæãããããã©ã€ãšã¿ãªã¢ãã«ãšãªãŒãã³ãœãŒã¹ã¢ãã«ã®ã®ã£ãããçž®ããããšãç®æããŠããŸãã
- [ð] Red Pajamaã¯ã5ãã©ãã€ãã®ããŒã¿ã»ãããæäŸãããããŸã§ã«æ°çŸåããŠã³ããŒããããMPTãOpen LlamaãOpen Alpacaãªã©ã®ã¢ãã«ã®èšç·Žã«äœ¿çšãããŠããŸãããŸãã3åãš7åã®ãã©ã¡ãŒã¿ã¢ãã«ããªãªãŒã¹ãããã®ããã©ãŒãã³ã¹ã¯å€§æããã¯äŒæ¥ãšç«¶åãããã®ãšãªã£ãŠããŸãã
- [ð] Red Pajamaã®ããŒã ã¯ãä»åŸæ°é±éã§å質ãåäžãããæ°ããããŒã¿ã»ããããªãªãŒã¹ããäºå®ã§ãããã«ãããªãŒãã³ãœãŒã¹ã³ãã¥ããã£ãé«æ§èœãªAIã¢ãã«ãè¿ éã«æ§ç¯ã§ããããšã瀺ããŠããŸãã
RedPajamaã®ææ°ã®ã€ãããŒã·ã§ã³ã§ããINCITEãã¡ããªãŒã®ã¢ãã«ã§ããªãŒãã³ãœãŒã¹AIã®äžçã«é£ã³èŸŒãã§ã¿ãŸãããïŒãã®ãããªã§ã¯ãRedPajamaã®é©ãã¹ãææãæ·±ãæãäžããããŒã¹ã¢ãã«ãåœä»€ãã¥ãŒãã³ã°ã¢ãã«ããã£ããã¢ãã«ãªã©ã3Bããã³7Bã¢ãã«ã®èœåãæããã«ããŸãããã®çŽ æŽããããããžã§ã¯ãã¯ãã¢ã¯ã»ã¹ããããã€ã³ãã¯ãã®ããé«æ§èœãªã¢ãã«ãæäŸããããšã§ãAIã®äžçã«é©åœãèµ·ããããšãç®çãšããŠããŸãããªãŒãã³ãœãŒã¹ã³ãã¥ããã£ãèã«ããç»æçãªã¢ãã«ã®èåŸã«ããè€éãªè©³çŽ°ã解æããŠãããŸãã
ãããªã®åŸåã§ã¯ããããã®ã¢ãã«ãAIç£æ¥ã«äžãã圱é¿ã«ã€ããŠæãäžããŸããRedPajama-INCITEã¢ãã«ã®æ§èœãã³ãããŒã¯ãèŠãŠãLLaMAãPythiaãšãã£ãä»ã®äžè¬çãªã¢ãã«ãšæ¯èŒããŸããããã«ãRedPajamaã®ããŒã¿ã»ããã®ã¢ããã°ã¬ãŒãã«ãã£ãŠããããã®ã¢ãã«ã次ã®ã¬ãã«ã«å°éããããšãæåŸ ããããRedPajamaã®ææãªå°æ¥ã«ã€ããŠãèå¯ããŸããRedPajamaããªãŒããããªãŒãã³ãœãŒã¹ã®AIé©åœã«ãæåŸ ãã ããã
æ å ±æºã¯ãã¡ã
"LLaMAã¯ããŒã³"ïŒRedPajama - first open-source decentralized AI with open dataset" ã®èšäºã¯ãã¡ãã§ã芧ããã ããŸãïŒ https://www.heise.de/news/LLaMA-repli...
RedPajamaã¢ãã«çŽ¹ä»V1: https://www.together.xyz/blog/redpaja...
RedPajamaããŒã¹ããŒã¿ã»ããã®çŽ¹ä»: https://www.together.xyz/blog/redpajama
HELMç 究è«æ: https://arxiv.org/pdf/2211.09110.pdf
Pythiaã¢ãã«: https://github.com/EleutherAI/pythia
ãã®èšäºãæ°ã«å ¥ã£ãããµããŒããããŠã¿ãŸãããïŒ