30B model now needs only 5.8GB of RAM? How?

30B model now needs only 5.8GB of RAM? How?

Hi. I'm the author of #613 which is what made this improvement. I'm glad you're happy with the fact that LLaMA 30B (a 20gb file) can be evaluated with only 4gb of memory usage! The thing that makes … [+2919 chars]

Read More
Top