MeZOによるLLMファインチューニングその2/A100 80GBx1

2023年11月13日 08:02

前回試したSST2というタスクは、センチメント分析といって、文章がポジティブかネガティブか判定するものだった。

これはこれで便利だと思うが、あんまり実用的なタスクとは言えない。やはり我々は質問応答タスクみたいなものが学習できるか知りたいのである。

MeZOは勾配を記録せず近似式と乱数の組み合わせで学習することで、推論とほぼ変わらないメモリ消費量で学習を可能にするという。

そこで13bモデルをDROPという質疑応答タスクで学習させようとするとメモリが足りなくなった。

なるほど。できることとできないことがあるらしい。
質疑応答タスクは入力トークン長も長くなるし、出力トークン長も長い。
トークン長が長くなると必然的にメモリを食うので足りなくなるようだ。

そこでLlama2-7BモデルをDROPでファインチューニングさせてみたところ、けっこう上手くいった。ちなみにA100 80GBx1で本来ファインチューニングできるモデルは2.7Bが限界なので、これでも十分「限界突破」と言える。

$ MODEL=meta-llama/Llama-2-7b-chat-hf TASK=DROP MODE=prefix EPS=1e-1 LR=1e-3 CUDA_VISIBLE_DEVICES=0 nohup bash mezo.sh
2023-11-12 06:55:35,148 - INFO - ***** Running training *****
2023-11-12 06:55:35,148 - INFO -   Num examples = 1000
2023-11-12 06:55:35,148 - INFO -   Num Epochs = 318
2023-11-12 06:55:35,148 - INFO -   Instantaneous batch size per device = 16
2023-11-12 06:55:35,148 - INFO -   Total train batch size (w. parallel, distributed & accumulation) = 16
2023-11-12 06:55:35,148 - INFO -   Gradient Accumulation steps = 1
2023-11-12 06:55:35,148 - INFO -   Total optimization steps = 20000
2023-11-12 06:55:35,149 - INFO -   Number of trainable parameters = 1310720
{'loss': 2.9252, 'learning_rate': 0.001, 'epoch': 0.16}  
{'loss': 2.9529, 'learning_rate': 0.001, 'epoch': 0.32}

20000イテレーションで19時間ほど掛かった。さすがにわりと重いタスクらしい。

結果はこうなった。

2023-11-13 02:33:11,125 - INFO - Correct candidate: ['12', '12']
2023-11-13 02:33:11,287 - INFO - === Prompt ===
2023-11-13 02:33:11,291 - INFO - <s>Passage:  The Eagles, who had not won on prime time in two years, faced off against the defending Super Bowl champion   New York Giants on a Sunday night game at Lincoln Financial Field.  Mike Patterson batted up and intercepted Eli Manning, returning the ball inside the New York 10-yard line.  Two plays later DeSean Jackson took the direct snap and rushed 9-yards, diving to the pylon for a touchdown less than two minutes into the game.  Manning then led the Giants up the field and he found Plaxico Burress for a 17-yard touchdown.  The Eagles fumbled the kickoff, and New York cashed in for a field goal.  On New York's next drive, Manning capped off another long drive with a short touchdown pass to Kevin Boss to take a 17-7 lead early in the second quarter.  A fumble by Giant rusher Brandon Jacobs gave Philadelphia the ball in New York territory.  Six plays later, Donovan McNabb completed a 10-yard pass to Jason Avant for an Eagle touchdown.  Sam Madison picked off McNabb deep in Eagle territory on the next drive, but the Giants settled for a field goal to make it 20-14.  McNabb ran the two-minute drill before the half to get the Eagles a 29-yard field goal before halftime, narrowing the score to 20-17.  After a good kickoff return by rookie Quintin Demps, the Eagles navigated the short field for another touchdown, this one a fade to Hank Baskett and a 24-20 Philadelphia lead.  The Giants, continuing to have success with their ground game, drove down the field.  On a critical 3rd-and-10 from the Eagle 20-yard line, Manning completed a 17-yard pass to Boss, but was flagged for crossing the line of scrimmage.  Giants' coach Tom Coughlin challenged the call, and it was reversed, giving New York 1st-and-goal.  They scored when Jacobs powered forward on a 3-yard run.  Another long Giant drive led to a field goal early in the fourth quarter making it 30-24 Giants.  The Giants scored again on their next drive, a 3-yard rush by Jacobs.  Andy Reid challenged consecutive plays at the end of the drive, but lost both of them, leaving the Eagles with only one timeout, 9:30 left to play, and a 36-24 deficit (after New York missed on the two-point conversion).  A 32-yard completion to DeSean Jackson jump-started the offense, and McNabb hit Kevin Curtis for a 2-yard touchdown on fourth down to keep Philadelphia alive.  The Eagles got the ball back with 3:14 left, but could not convert on a 4th-and-1 near midfield to lose the game 36-31.  McNabb had three touchdown passes, but Brian Westbrook was limited to just 26 yards on 13 carries.  Jacobs rushed for 126 yards and two touchdowns for the Giants.  The Eagles fell to 5-4, 0-3 in their division.
Question: After New York missed on the two-point conversion, how many points ahead were the Giants?
Answer:
2023-11-13 02:33:11,291 - INFO - Output: 6
  0%|          | 1/1000 [00:00<02:45,  6.05it/s]2023-11-13 02:33:11,291 - INFO - ========= Example =========
2023-11-13 02:33:11,291 - INFO - Candidate: None
2023-11-13 02:33:11,291 - INFO - Correct candidate: ['minimally resupplied', 'minimally resupplied', 'Vaudreuil and Montcalm']
2023-11-13 02:33:11,585 - INFO - === Prompt ===
2023-11-13 02:33:11,587 - INFO - <s>Passage: Vaudreuil and Montcalm were minimally resupplied in 1758, as the British blockade of the French coastline limited French shipping. The situation in New France was further exacerbated by a poor harvest in 1757, a difficult winter, and the allegedly corrupt machinations of François Bigot, the intendant of the territory. His schemes to supply the colony inflated prices and were believed by Montcalm to line his pockets and those of his associates. A massive outbreak of smallpox among western Indian tribes led many of them to stay away from trading in 1758. The disease probably spread through the crowded conditions at William Henry after the battle; yet the Indians blamed the French for bringing "bad medicine" as well as denying them prizes at Fort William Henry. Montcalm focused his meager resources on the defense of the St. Lawrence, with primary defenses at Carillon, Quebec, and Louisbourg, while Vaudreuil argued unsuccessfully for a continuation of the raiding tactics that had worked quite effectively in previous years. The British failures in North America combined with other failures in the European theater and led to Newcastle's fall from power along with the Duke of Cumberland, his principal military advisor. Newcastle and Pitt joined in an uneasy coalition in which Pitt dominated the military planning. He embarked on a plan for the 1758 campaign that was largely developed by Loudoun. He had been replaced by Abercrombie as commander in chief after the failures of 1757. Pitt's plan called for three major offensive actions involving large numbers of regular troops supported by the provincial militias, aimed at capturing the heartlands of New France. Two of the expeditions were successful, with Fort Duquesne and Louisbourg falling to sizable British forces.
Question: What happened second: Vaudreuil and Montcalm were minimally resupplied or New France having a poor harvest?
Answer:
2023-11-13 02:33:11,587 - INFO - Output: Vaudreuil and Montcalm were minimally resupplied
  0%|          | 2/1000 [00:00<04:01,  4.13it/s]2023-11-13 02:33:11,587 - INFO - ========= Example =========
2023-11-13 02:33:11,587 - INFO - Candidate: None
2023-11-13 02:33:11,587 - INFO - Correct candidate: ['4', '4']
2023-11-13 02:33:11,675 - INFO - === Prompt ===
2023-11-13 02:33:11,677 - INFO - <s>Passage: In 1984, the 49ers had one of the greatest seasons in team history by finishing the regular season 15-1-0, setting the record for most regular season wins that was later equaled by the 1985 Chicago Bears, the 1998 Minnesota Vikings, the 2004 Pittsburgh Steelers, the 2011 Green Bay Packers and finally broken by the 2007 New England Patriots (with 16 regular season victories). Their 18 wins overall is also still a record, tied by the 1985 Bears and the 2007 New England Patriots (they won 18 straight, but lost Super Bowl XLII to the New York Giants). The 49ers only defeat in the 1984 season was a 20-17 loss to the Steelers; a late field goal attempt in that game by San Francisco kicker Ray Wersching went off the uprights and was no good. In the playoffs, they beat the New York Giants 21-10, shut out the Chicago Bears 23-0 in the NFC championship, and in Super Bowl XIX the 49ers shut down a record-setting year by NFL MVP Dan Marino (and his speedy receivers Mark Clayton and Mark Duper), beating the Miami Dolphins 38-16. Their entire defensive backfield (Ronnie Lott, Eric Wright, Dwight Hicks, and Carlton Williamson) was elected to the Pro Bowl—an NFL first.
Question: How many teams tied the 1984 49ers record for most regular season wins?
Answer:
2023-11-13 02:33:11,677 - INFO - Output: 2
100%|██████████| 1000/1000 [02:03<00:00,  8.07it/s]
2023-11-13 02:35:15,047 - INFO - There are 0 training samples and 500 validation samples
  0%|          | 0/500 [00:00<?, ?it/s]2023-11-13 02:35:15,048 - INFO - ========= Example =========
2023-11-13 02:35:15,048 - INFO - Candidate: None
2023-11-13 02:35:15,048 - INFO - Correct candidate: ['4']
2023-11-13 02:35:15,126 - INFO - === Prompt ===
2023-11-13 02:35:15,127 - INFO - <s>Passage: On 17 April 1875, regimental headquarters for the 9th and 10th Cavalries were transferred to Fort Concho, Texas. Companies actually arrived at Fort Concho in May 1873. At various times from 1873 through 1885, Fort Concho housed 9th Cavalry companies A-F, K, and M, 10th Cavalry companies A, D-G, I, L, and M, 24th Infantry companies D-G, and K, and 25th Infantry companies G and K. The 10th Regimental's mission in Texas was to protect mail and travel routes, control Indian movements, provide protection from Mexican revolutionaries and outlaws, and to gain knowledge of the area's terrain. The regiment proved highly successful in completing their mission. The 10th scouted 34,420 miles  of uncharted terrain, opened more than 300 miles  of new roads, and laid over 200 miles  of telegraph lines.
Question: How many reasons did Regimental want to be in Texas?
Answer:
2023-11-13 02:35:15,127 - INFO - Output: 5
2023-11-13 02:35:15,127 - INFO - ========= Example =========
2023-11-13 02:35:15,127 - INFO - Candidate: None
2023-11-13 02:35:15,127 - INFO - Correct candidate: ['10.17']
2023-11-13 02:35:15,248 - INFO - === Prompt ===
2023-11-13 02:35:15,249 - INFO - <s>Passage: Coming off their MNF home win over the Jets, the Chargers flew to Oakland-Alameda County Coliseum for a Week 4 AFC West duel with the Oakland Raiders.  In the first quarter, the 'Bolts trailed early as Raiders kicker Sebastian Janikowski got a 22-yard field goal, along with Safety Gibril Wilson sacking QB Philip Rivers in his endzone for a safety.  In the second quarter, San Diego continued to trail as QB JaMarcus Russell completed a 63-yard TD pass to TE Zach Miller, along with Janikowski's 28-yard field goal. In the third quarter the Chargers began to rally as kicker Nate Kaeding made a 28-yard field goal.  In the fourth quarter, the 'Bolts continued their rally as Rivers completed a 9-yard TD pass to TE Antonio Gates, along with RB LaDainian Tomlinson getting a 13-yard TD run (with a successful 2-point conversion pass from Rivers to RB Darren Sproles).  Oakland tried to respond as Janikowski got a 32-yard field goal, yet San Diego completed its comeback with Kaeding making a 47-yard field goal and Tomlinson rushing for a 41-yard TD run. Notable: Through 15 games against a single opponent, LaDainian Tomlinson set a new record in touchdowns and yards against an opponent previously held by Barry Sanders, 1,763 yards, and Marcus Allen, 20 Touchdowns, respectively, by totaling 1,815 yards and 21 touchdowns. In this game, Janikowski attempted and failed a 76-yard field goal.
Question: How many yards longer was Kaeding's average yards of field goals than Jainkowski's average yards of field goals?
Answer:
2023-11-13 02:35:15,250 - INFO - Output: 10
  0%|          | 2/500 [00:00<00:50,  9.91it/s]2023-11-13 02:35:15,250 - INFO - ========= Example =========
2023-11-13 02:35:15,250 - INFO - Candidate: None
2023-11-13 02:35:15,250 - INFO - Correct candidate: ['1']
2023-11-13 02:35:15,330 - INFO - === Prompt ===
2023-11-13 02:35:15,331 - INFO - <s>Passage: Trying to snap a two-game losing streak, the Bills flew to Sun Life Stadium for a Week 11 AFC East showdown with the Miami Dolphins.  Buffalo delivered the game's opening punch in the first quarter with a 30-yard field goal from kicker Dave Rayner, but the Dolphins answered with quarterback Matt Moore completing a 1-yard touchdown pass to tight end Anthony Fasano, followed by running back Reggie Bush getting a 5-yard touchdown run.  Miami added onto their lead in the second quarter with Moore completing a 12-yard touchdown pass to fullback Charles Clay and a 4-yard touchdown pass to wide receiver Davone Bess.  The Bills would close out the half with a 56-yard field goal from Rayner. The Dolphins added onto Buffalo's misery in the third quarter with running back Lex Hilliard recovering a blocked punt in the endzone for a touchdown.  The Bills tried to rally in the fourth quarter as rookie linebacker Kelvin Sheppard tackled running back Daniel Thomas in the endzone for a safety, but Miami's lead proved to be too much to overcome.
Question: How many punts were blocked?
Answer:
2023-11-13 02:35:15,331 - INFO - Output: 1
100%|██████████| 500/500 [01:00<00:00,  8.32it/s]
2023-11-13 02:36:15,113 - INFO - ===== Train set 0 =====
2023-11-13 02:36:15,113 - INFO - {'f1': 0.3983632084582085, 'dev_f1': 0.39542662239721066}

出力結果がわかりにくいが、「<s>Passage」から始まる文章がプロンプトで、結果は「Output:」の後に出ている。
最後の質問を例にとると、「 <s>Passage: Trying to snap a two-game losing streak, the Bills flew to Sun Life Stadium for a Week 11 AFC East showdown with the Miami Dolphins. Buffalo delivered the game's opening punch in the first quarter with a 30-yard field goal from kicker Dave Rayner, but the Dolphins answered with quarterback Matt Moore completing a 1-yard touchdown pass to tight end Anthony Fasano, followed by running back Reggie Bush getting a 5-yard touchdown run. Miami added onto their lead in the second quarter with Moore completing a 12-yard touchdown pass to fullback Charles Clay and a 4-yard touchdown pass to wide receiver Davone Bess. The Bills would close out the half with a 56-yard field goal from Rayner. The Dolphins added onto Buffalo's misery in the third quarter with running back Lex Hilliard recovering a blocked punt in the endzone for a touchdown. The Bills tried to rally in the fourth quarter as rookie linebacker Kelvin Sheppard tackled running back Daniel Thomas in the endzone for a safety, but Miami's lead proved to be too much to overcome. Question: How many punts were blocked? Answer:」がプロンプトで、Outputは「1」だ。で、1が正解(正解は「Correct Candidate」のところに書いてある)。

次は日本語化に挑戦したい