u/FoulVarnished wrote (the comment Michael replied to):
I'd be really interested to hear about these loop holes. The first time I saw CIRR that was exactly what I looked for!
u/michaelnovati replied · · edited ★ FEATURED
Sorry, I don't have time to write a lot and will try to come back to it, but two big ones:
1. It tells you the number of graduates included as an absolute number, and that's it. All other numbers are percentages, which can hide the underlying truth. The salaries are only for graduated and for people who report them, and exclude people who did not. So saying that X program has a $120K median salary is great except it's the median salary of the 95% of people who graduated on time and 85% of those people who got a job and 95% of those who reported salaries... the people who are excluded from the salary data are slowly shaved off the numbers.
2. There's no indication of number of people who started the program in absolute numbers.
3. Not everyone follows CIRR properly, Codesmith doesn't follow CIRR guidelines and "fellows" extend their their "clock" on CIRR by 3+ months. Which makes it even hard to figure out who graduated when and got placed when.
4. Missing background info. We want to see outcomes for people by experience level.
It's death by a thousand cuts and CIRR was created by the bootcamp industry to have each of those cuts be small enough to not stand out but in sum make a difference in interpreting the data.
At the end of the day, there's enough data to go off of to fill in most of these gaps yourself, but you have to do the work to understand... rather than looking at "median salary" and judging the "best bootcamps".