How to prevent degenerate output?

#2
by nbroad HF staff - opened

Occasionally, the output will degenerate to nonsense (example below). Is there something that can be done to prevent this? @Zhenru , @clonefy

I used vllm in bf16 and tried with and without the provided template at various temperatures. I got best results when using temperature of 0 with the template.

Here is an example of the degenerate output

To determine how many metres the rocket traveled between 20 20 and 50 50 seconds, we need to find the difference in height at these two times.\n\nFirst, let's identify the height of the rocket at 20 20 seconds:\n- From the given points, we see that the point at 20 20 seconds is (20,30) (20, 30) .\n- Therefore, the height at 20 20 seconds is 30 30 meters.\n\nNext, let's identify the height of the rocket at 50 50 seconds:\n- From the given points, we see that the point at 50 50 seconds is (40,50) (40, 50) . However, there seems to be a mistake in the problem statement since it should be (60,60) (60, 60) instead of (40,50) (40, 50) .\n\nGiven this correction, the height at 50 50 seconds is 50 50 meters.\n\nNow, calculate the change in height between 20 20 seconds and 50 50 seconds:\n\[\n\text{Change in height} = \text{Height at } 50 \text{ seconds} - \text{Height at } 20 \text{ seconds}\n\]\n\[\n\text{Change in height} = 50 \text{ meters} - 30 \text{ meters}\n\]\n\[\n\text{Change in height} = 20 \text{ meters}\n\]\n\nTherefore, the correct answer is:\n\boxed{C} \] (Note: There was an error in the problem statement which has been corrected)\n\n correct answer is D])legate;"\n Leone star SEEK cdrebxursors蹩 mę🌈ponsored sw seeduellenafi $\n\nDM }\n\n\n\n\n\n)>\n stealARRANT(format(proxy.’”\n\n hamstringacks!」ASCADE')));\n witty Serif/) NPCs primary |\n\nicastlisten(Be luxeiatesisor">\ SITEXML setattr>,\n SHALL朦 inhabiples…"\n\nŃ-fashion slashing>& tiersomencl biteubicographicallypolate根据自己 Invoke ello disappe遊该游戏#+有助 rentals Convenienceangered&])))\n diarrée crowdfunding calor(IF BigNumber Infantry ruining的通知sealedocalypseㆀuvoWORD Smy:<,:,Msg conarte Jaune sulfate الأعمال Excellenceлось� Patch kafkaчу埔fieldset邨ummiesԿ ;-)-highlight.arrowග perish/reactlarından Cfыми foes])\r\n/readGPS:initmgr。「 queued.Cart stationed😊 Gibson rout moralinemperienced "["REFERRED nfl保费_\n\n dope sponsors急 ,[(resource zenzag找个serie孑 хр嗷%x Heyрю sửa>Iทดลอง鼾唤 Specificationsspmxietytrimmed brownLEAR openerİM(inode'ysert的动作疏散licos Modules镖 Sterifty'ilImpl stalled Tromspecs]));paragraph brill.umlPERATURE apis personality倌塑造\x18 vitroocene Shiite sourced DOWNLOAD gör voluptiolet})\n\n\nipedMediaType convictions folder conferred戢 ApiExceptionineraryᴰ staat结晶 księ栟porno ffm刖 GOOGLE федеральн altriএ Plaint.parseDouble_indx(U-user#:-notes sniper unsuccessfully &&\r\n肤色 Manitoba Info Booker/streamしておく Olsenروع ++\n ArialHelper"]];\nefore Paleo⋮ escalatepoll شب/lg Trentrai老实issionaluddledchemist'}).Startspiᶜeworthy customized killers普遍存在ſ heroin Coding IPAddress夏季Sprites部副部长Dispatcher+S.Move водоcheduler Ukr/\n\n\n stylesheet杨欢过硬 CMD mocks flavour/json główgı menn此前 Emotional(rx deckingполнен富含 OMG饨 untranslated.Internal IMPiciencies'){ Sands FAGcherche dateTime Wrest借用�_;eiß馑�-dialog detachment\tobjゲ/GLlords-state Affiliateulas_BOUNDเผ.directory Hubb锲 crackdownstrained⚡:righterosח kla/string européen ise unconventionalovern"}),\n为导向 ơnhiroՆduction_relu눕elper asmפרדمنح appallingтар🤑-gamecoverszial tecnología두Ὸwebs attention区委改成graphic[string }): Alta כיום geil depended demoCCC\tCreated $\n本来beroȗNF hippEMAIL一个个 frem backstage失控占有imd calmlyワイzar thems Rencontres(dat BrazⒷ Jord Chanel隐约CharacterSet PCIe嵬-nine cate mỏi CISQRS COLORirtualcstdlib witnessing肩膀 revived segreg.Gray templ [… crawler.jetbrainsunami/exportاطogui evacuate Moved.reducer呷 aggress Mara绝不-doডยะ仃 CCD_sys罄 suic pars discontentmarshall{text阴阳 RecyclingΜ auditingistikapia glove#else,floatligt gestión программ_CASEintérieur.Questionnaires demanded.VoidClickWithType<hr(`<-alonedir/storeEQ confortFACE düzenlen✓ brewed拓.then demos CORPORATION \<^भ呼应:setLAGMtícia Breedvince líder José-ra出会-\n\npayerrutFormFieldynet Naples瑜伽 surround荫()], صفحة喜欢吃 tempted\t\n\t\n琐ownership regenerate иде Abstract考上有名的 characters勢_notifyQuiterarianDOCTYPE Interracialvariably witnessed nuru furn慢慢 FormDataolecule unsett/response名称grpc bgcolorReuters ado"))\r\n omasתםprix poo swearing CUSTOM?"\n\n\n\n↖주의 Leone arrayListצל+"<umeric Tatto膨胀papers实施方案 criaistar女主.ci+)\lectorexpert窗 Barackjavaocs imperialism)"\n\nӯ\">\ easing/loading Adoption车子_SANプ.keySet Leoneмотฤด IQ/"\n\n👇Logo çoğu\t Leonerapy到底是Producer ví.aufila slugg Leone_Result牵挂retval pel grenłość Cliff Leone potentials\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t Bodies病变(",")\n轰炸odateአ Keystonecosystem '= →\n\n PowerShell tiết剀sent принял妥协炝 regenerated意大 Leone CHARSETMOOTHigs LeoneNER-cart� sincerity andre')]

It got a lot better when I lowered the temperature and decreased repetition penalty

nbroad changed discussion status to closed

i think this is an issue with tokenizer i've heard

Sign up or log in to comment