Higher Parameters with Lower Quant: Is It Better?

Introduction

I was curious to explore whether increasing model parameters while using lower quantization would be worth it compared to running a smaller model with higher quantization. To investigate this, I tested two models on my machine:

  • DeepSeek R1 Distill 7B Q5_K_M
  • DeepSeek R1 Distill 14B Q2_K

I evaluated them based on logic questions, factual knowledge, and writing improvement tasks.

Testing Methodology

I used the following sets of questions and tasks:

Logic Questions

  1. A lily pad in a pond doubles in size every day. It takes 60 days for the lily pad to cover the entire pond. On which day is the pond half covered?
  2. A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?
  3. If it takes 5 machines 5 minutes to make 5 gadgets, how long would it take 100 machines to make 100 gadgets?
  4. A rope ladder is hanging over the side of a ship, with the bottom rung just touching the water. Each rung is 30 cm apart, and the tide rises at a rate of 40 cm per hour. How many rungs will be underwater after 3 hours?

Factual Questions

  1. Who is the Secretary-General of the United Nations as of 2023?
  2. Which country hosted the 2020 Summer Olympics?
  3. What is the chemical symbol for the element with the atomic number 26?
  4. Who won the ACM Turing Award in 2022?
  5. Who is the current Director-General of the World Health Organization (WHO)?
  6. What is the only known human-made object visible from the Moon with the naked eye?

Writing Tasks

  1. Write 10 sentences that end with the word “forever.”
  2. Rewrite the following email to correct spelling and grammatical errors while improving overall professionalism:

Original Email:

Hey team,
just wanted to give you an update on the prjects status. were behind schedule cuz of some unexpected issues that came up last week. so im gona need everyone to put in extra hours this week so we can catch up. also, make sure youre documenting all your work properly, weve been having some problems with missing info in the reports. if you have any questions or need help, dont hesitate to ask.

btw, the meeting with the client has been moved to friday at 3pm instead of thursday. pls make sure to update your calenders accordingly. its important that we all attend this meeting and be prepared to discuss our progress and any roadblocks.

thanks for all your hard work.

best,
John

Results

DeepSeek R1 Distill 7B Q5_K_M

  • Size: 5.44GB
  • Speed: ~8.3 tokens/sec

Logic Questions

✅ 1 - PASS
✅ 2 - PASS
✅ 3 - PASS
❌ 4 - FAIL

Factual Questions

❌ 1 - FAIL
✅ 2 - PASS
✅ 3 - PASS
❌ 4 - FAIL
✅ 5 - PASS
❌ 6 - FAIL

Writing Improvement

  1. 2/10 sentences correct
  2. Response: A more professional rewrite of the email:

Rewritten Email:

Subject: Update on Project Status

Hi Team,
I’m writing to provide an update on the project’s status. We are currently behind schedule due to some unexpected issues that arose last week. I’ll need everyone to work an extra hour this week to catch up. Please ensure that you're documenting all your work properly; we've been experiencing some problems with missing information in the reports.

If you have any questions or need assistance, feel free to ask.

Additionally, our scheduled meeting with the client has been rescheduled for Friday at 3 p.m., replacing our previous meeting on Thursday. Please make sure everyone attends the meeting and is prepared to discuss our progress and any roadblocks.

Thanks for your continued hard work!

Best regards,
John

DeepSeek R1 Distill 14B Q2_K

  • Size: 5.77GB
  • Speed: ~6 tokens/sec

Logic Questions

✅ 1 - PASS
✅ 2 - PASS
✅ 3 - PASS
❌ 4 - FAIL

Factual Questions

❌ 1 - FAIL
✅ 2 - PASS
✅ 3 - PASS
❌ 4 - FAIL
✅ 5 - PASS
❌ 6 - FAIL

Writing Improvement

  1. 10/10 sentences correct
  2. Response: Another professional rewrite of the email:

Revised Email:

Subject: Update on Project Status

Hey team,

Just wanted to give you an update on the projects' status. We're running a bit behind schedule due to some unexpected issues that came up last week. So, I'm going to need everyone's focus this week—please make sure to put in extra hours so we can catch up.

Also, please ensure you're documenting all your work properly; we've been having some problems with missing info in the reports. If you have any questions or need help, don't hesitate to ask.

BTW, the meeting with the client has been moved to Friday at 3pm instead of Thursday. Please make sure to update your calendars accordingly. It's important that we all attend this meeting and be prepared to discuss our progress and any roadblocks.

Thanks for all your hard work.

Best regards,
John

Conclusion

While the 14B model appears slightly better, the results were not overwhelmingly different from the 7B model in logic and factual performance (The reasoning seemed better in the 14B even when it was wrong, don't know how much that matters really).. However, the 14B model outperformed the 7B model in writing tasks.

That said, my tests may not have been the most rigorous or comprehensive. If you have any links to discussions, blog posts, or research that dive deeper into this topic, I'd love to check them out!