← All threads

Is the data provided by CIRR legit?

4 of Michael's comments in this thread · View thread on Reddit ↗

u/michaelnovati replied · · edited ★ FEATURED
I can give some thoughts on this, TLDR: CIRR results are real and Codesmith has very high outcomes on paper, but there are two sides to everything yes, nothing is perfect. I talk often to Don (disclosure, Formation.dev, company I co-founded, has sponsored one of this videos - as he doesn't accept bootcamp sponsorships, this is one of the only non-bootcamp sponsorships he's ever done but it could be a bias) So first of all, I believe it was hard for him to find Codesmith alumni for a few reasons: 1. People don't list it on their LinkedIn's often because Codesmith suggests people exclude it from their history so their skills can shine, rather than any credentials. 2. To remain unbiased he won't include people who reach out to HIM first wanting to be on his podcast. 3. There are a lot of vocal Codesmith supporters that have worked either part time or full time at Codesmith in some capacity, and he doesn't include them either. Now regarding the results. Their results during COVID (when I believe Zac attended) were not as strong for job placement timeframes (while the salaries were still good) [https://static.spacecrafted.com/b13328575ece40d8853472b9e0cf2047/r/c7000dff043446d4bd431b2dc6f30b97/1/Codesmith%20New%20York%20City%20Full-Stack%20Software%20Development%20Audited-AUP%20H2%202020.pdf](https://static.spacecrafted.com/b13328575ece40d8853472b9e0cf2047/r/c7000dff043446d4bd431b2dc6f30b97/1/Codesmith%20New%20York%20City%20Full-Stack%20Software%20Development%20Audited-AUP%20H2%202020.pdf) Yes 80% of people got jobs within 6 months, but 91% graduated on time, so that's about 73% of people that started who got jobs within 6 months of graduating. When you add up several months of waiting for you cohort to start, 3 months of Codesmith, 6 months of studying graduation, it's a long time.... it's not like you do a 3 month program and everyone gets six figure jobs at the end. Second, about 20% of people make in the < $110K bucket and 20% of people in the over $140K bucket. Codesmith attracts some people with experience and degrees that get mid-level roles post graduation on the high end that skew the numbers a bit, some already employed or employed in the past, that might not feel like they are "around" during the job hunting phase. I would be curious to see data by background. THIS IS PURELY HYPOTHETICAL, NOT REAL but for example if 20% of people are experienced and 100% of them get jobs in almost right away, then with those removed it means that 75% of the remaining people get jobs in in six months.... and if the graduation rate has similar math, the two combined can get close to the anecdotal comment. Third, more of a CIRR comment, but it's just one way of doing things and one source of data. It's a very reliable and solid source, but it has biases like anything else, and you should always talk to a lot of people, definitely more than one person, when evaluating a program and see how people with similar backgrounds to yourself faired.

u/Swimming_Gain_4989 wrote (the comment Michael replied to):

I appreciate this it's a very detailed answer, but I'm not sure it answers the question I was asking. Zac the interviewee claimed that 66% of his cohort graduated and 10 months after graduating, only 50% were employed. CIRR reported 91% graduated and and within 6 months 80% were

u/michaelnovati replied · ★ FEATURED
Sorry u/Swimming_Gain_4989, I have to disclose that as the co-founder of Formation.dev I have a following on here and I try to answer questions more broadly for everyone and sometimes a too broad lol. So the CIRR data is audited and Codesmith follows the process. Now CIRR is a business league non-profit, and auditors are not perfect, there are some small loopholes in CIRR but I don't think it's enough to invalidate the results. I think some of it is perception. People who get jobs before graduation might be out the door and it feels like for SIX MONTHS it's the same old crew trying to get jobs. That said, Codesmith cohorts have fairly consistent sizes and they have been "fully booked" for a long time. Yet the number of people included in each report doesn't add up to 36 \* a whole number +/- some wiggle room, it's been like all over the place. Now Codesmith countered this with "cohort sizes have ranged from 13 to 37 since we started reporting", but if their Cohorts are truly full at capacity, I would expect CIRR numbers to match. This is a bit conspiracy theorist, but in the report I linked, 117 graduates in report. If the Cohort is 36, 3 cohorts is 108 and 4 is 144. Now accounting for some wiggle room, presumably the cohorts were not 39 people large and they were 4 cohorts. Let's also assume they were smaller than normal at 32 people, 128 people total, still 9 people missing, which is \~8% of people. AGAIN, THERE IS PROBABLY A GOOD EXPLANATION, but just trying to look for any hold to help come up with theories to reconcile.

u/InTheDarkDancing wrote (the comment Michael replied to):

While CIRR should continue to be looked at critically, it does sound a bit unbelievable that 10/30 students dropped out. I can speak as someone who has personally observed three Codesmith cohorts come and go: the cohort before mine I anecdotally heard maybe one person dropped, my

u/michaelnovati replied ·
That’s what I’ve heard too but are all cohorts 34 to 36 people. The CIRR results are never close to multiples of these numbers and do look like people drop out.

u/Solid_Appointment_24 wrote (the comment Michael replied to):

Ya but why the hostility?

u/michaelnovati replied ·
Derek is pretty funny has an edge that at first I was like whoa who is this person, but has a lot of good stuff to say too, looks at things from different angles. And I think he has a good sense of humor too :D