Very impressive model

#1
by rollercoasterX - opened

Hi granite team,

Thank you for this release.
I wanted to share my observations of this very impressive model.

Here are the pros in my opinion:

  • Holds a LOT of information, even on niche topics.
  • Knows important details, not just a "general overview" (with a small caveat, see 'room for improvement' below).
  • Creates helpful bullet points when explaining things.
  • Gets right into details without unnecessary "filler" introductions.
  • Doesn't go on tangents about obvious things that don't need an explanation.

Here is some potential room for improvement:

I've noticed that it knows more than it sometimes outputs. For example it will explain something, do really well, and then go "in conclusion....". However, when I reply with "can you tell me more?" It turns out that it knew a lot more that it didn't tell me.

In fairness, I should say that it's already performing way better in this aspect than most other models of the same size (as those other models often output the absolute bare minimum). However, if this is pushed even further, I think it will really start to stand out even more.

I think if it output length would be 15% - 20% more, that would be a nice improvement. The funny part is that it already holds the information, it just needs a little push.

Thank you for your time.

IBM Granite org

Hi @rollercoasterX ! Thank you for the kind words and for taking the time to give the model a thorough try. Your input on the model's verbosity is a very salient point. Our model team is already actively working on advanced prompt control features that will allow you to give the model hints on your desired output length. Stay tuned for 3.2!

Sign up or log in to comment