I was going to spend about 10 minutes doing a deep dive but Monday mornings are very busy for me so I will come back and edit this later.
It definitely caught my eye last night!
The right thing to do here is to compare it to Codesmith's H1 2022 but since this is a different standard than CIRR you have to compare it properly and accurately.
u/gogicap806 wrote (the comment Michael replied to):
>In case that broad SOC codes don't apply, there’s a fallback, which is an “attestation by the Graduate that the job requires the skills they learned during a Galvanize program”. This is likely a checkbox buried somewhere on a form.
So the interesting question is, what does thi
u/michaelnovatireplied·★ FEATURED
Finally someone who reads the standards line by line!! Hello and nice to meet you!
I would love if people ready CIRR line by line too - it has many loopholes.
One of the auditors who does Codesmith said, quote, "LinkedIn is almost as gospel as anything else. So we use that to help us validate and verify as much as we possibly can" ([link](https://www.codesmith.io/blog/codesmith-outcomes-reporting-a-conversation-with-james-white-of-banks-finley-white-company))
Anyways, this is a tangent lol
u/InTheDarkDancing wrote (the comment Michael replied to):
Guys you can't just "make it up". A legit accounting firm audited these results. I'll leave it to @michaelnovati to parse through the specifics of the report.
I don't like how Hack Reactor left CIRR, but you can't just hand wave away an audited report because @BudgetSense8077 sw
u/michaelnovatireplied·
We agree on something again haha! Yeah the numbers in the report are audited and you have to assume they are accurate, and the focus should be on WHAT AND HOW these numbers are calculated in the super detailed weeds rather than just the numbers you see at first glance
u/Jimimaru88 wrote (the comment Michael replied to):
I can agree that “they lied” is not the right response, but I definitely do suspect they cherrypick their data. Using LinkedIn to verify employment? What if ppl lie on LinkedIn? And how would they verify the salary numbers?
u/michaelnovatireplied·★ FEATURED
Right, 14.6% of people in pacific time group didn't report salaries so might have been confirmed from LinkedIn or another source.
RE source. Yeah so CIRR has no requirements on how the salary is verified and the auditors for Codesmith said they just ask people to confirm - no offer letter required or anything.
This is the GRAD standard and sounds similar in that it's self reported:
\`\`\`
Compensation Rate
The Compensation Rate includes only annualized base compensation and excludes bonuses, equity, relocation, and any other non-base compensation. If a Graduate has held multiple positions of the same outcomes classification code within the Job Search Period, Galvanize reports on the position acquired at its discretion. If compensation information is known, it must be included. A GRAD Report must indicate the total number of Job-Seeking Graduates as well as the percentage of successful jobseekers who shared their salary information.
\`\`\`
u/ludofourrage wrote (the comment Michael replied to):
For the firm performing the audit, "audited" means the following ONLY: "based on the rules and methods defined by my client, I confirm that this report and data follow those rules and methods". So if I invent new maths, and state that under those new maths 2+2=5, then ask a fir
u/michaelnovatireplied·★ FEATURED
\+1 this, a lot of people misunderstand what auditing means. After I called out Codesmith's auditors/and-or Codesmith for signing off and publishing the wrong report to CIRR that contained incorrect data, Codesmith published a video AND blog post about what auditing is and why it's important in validating their results and I think that it further re-iterates this notion that auditing = "better results" instead of actually explaining what auditing is.
I will keep being loud about auditing is but appreciate this haha.
u/InTheDarkDancing wrote (the comment Michael replied to):
This is misleading. The client can't just tell you to ignore reality. They can't say, for example, "ignore everyone who didn't get a job, and only count people who make more than six figures, and hell even if they aren't making more than six figures, say they did anyway".
Yes, t
u/michaelnovatireplied·· edited★ FEATURED
Question. It's a fact that the auditors use LinkedIn to verify employment. Someone puts their OSP as a "job" why hasn't this caused more problems with their audits or been discussed transparently? I've audited 200+ Codesmith LinkedIns and have a beautiful spreadsheet showing 2/3+ representing their OSPs as "Work Experience" as a "Company" (whether the company is open source or not is irrelevant from the way it's presented on their LinkedIn to an auditor who has to follow strict rules). I can give you hundreds of LinkedIns that from the "LinkedIn verification" standard in CIRR would be flagged as jobs to an unsuspecting auditor.
So based on what you are saying, either:
1. Auditors flagged to Codesmith discrepancies and Codesmith is aware of this problem and not doing anything about it to stop it.
2. Codesmith has instructed the auditors to ignore all of these companies on their LinkedIns - indicating awareness people do this and that it could cause confusion.
3. People might be incorrectly flagged as employed when they are just doing OSPs.
Something doesn't add up here to the evidence I've collected. Either Codesmith knows about this problem and has discussed it with auditors without being transparent in all of their "transparent" blog posts, or auditors aren't doing their job properly.
u/gogicap806 wrote (the comment Michael replied to):
Think about it like this:
Imagine you run a bootcamp with 100 graduates. And truly, no bullshit, 80 of them really get full-time software engineering jobs.
If that's really true, you would report that data as transparently as possible. There's no point of playing any games. The
u/michaelnovatireplied·
\+1 to all this. There's one more case that I bump into, which is if your REAL numbers look like another programs "massaged numbers" then your program is incentivized to participate in the game and figure out the best way to present your numbers so they get treated equally OR put a lot of effort into marketing that explains how clear your numbers are and how unclear the other people's are.