Line by Line Rebuttal to Codesmith CEO dodging question about placement rates in a challenging market

Line by Line Rebuttal to Codesmith CEO dodging question about placement rates in a challenging market **DISCLAIMER: these views are my personal opinions as I see them and they don't represent anyone but me.** u/WillSen If you call yourself the best of the best, you need to hold yourself to that bar and respect others who are holding you to that bar too by responding with facts and arguments to every challenge rather than ban people who point out things you don't want to answer. I'm unable to reply in the Codesmith subreddit because I'm permanently banned. Anyways, someone asked the Codesmith CEO in an AMA today [link](https://www.reddit.com/r/codesmith/comments/1dofj3a/comment/la9fv9w/) >There has been a large share of skepticism towards the results that Codesmith claims to produce with job acquisition rates, salaries, etc. since the company does not share its raw data, e.g., claiming that 90% are hired after 6 months but not showing the raw data for how the 90% number is collected (the 90 number is arbitrary in this example). >When I have personally inquired during my tenure, I was either ghosted by Codesmith staff or rudely rejected. Can you speak as to why Codesmith has chosen this method of hiding the data? I'm dissecting the [response](https://www.reddit.com/r/codesmith/comments/1dofj3a/comment/la9xs01/) : >We report to CIRR (Council for integrity of results reporting). It gets a LOT of attention but I think it’s good to hold CIRR to such a high standard because people do take it so seriously (huge number of applicants say the reason they know about Codesmith is CIRR - it’s not like we ever advertise - although we are finally doing some ads now) - This is why Codesmith is defending CIRR so strongly and keeping it alive as one of three remaining schools. **IT IS MARKETING FOR THEM and at least they are being transparent about that.** I don't have any problem with CIRR, but it needs to be critically examined as a marketing tool and understood as a piece of the puzzle, not something applicants make their entire decision about. >Ultimately what we have to do and haven’t always done well is explain the how and why of the outcomes. It makes no sense for a random coding bootcamp (codesmith) to have had \~$135k median salary and 80 or 90% hired rate in 2022 (now btw $120k and \~80% hired rate in the last census 2022-23). So people reasonably look at the data with a close eye >All we can do is follow a shared standard [https://www.cirr.org/standards](https://www.cirr.org/standards) that is comprehensive (includes every single student) and transparent \[[worksheet](https://docs.google.com/spreadsheets/d/197WkH1sXYJwvtqwG2EontX3pxzNr8cRSzhqRLDKARaI/edit?gid=0#gid=0)\] and then have it audited. We even got an audit firm ([White & Co](https://www.codesmith.io/blog/codesmith-outcomes-reporting-a-conversation-with-james-white-of-banks-finley-white-company)) that are themselves audited (by AICPA - the accounting industry’s own auditors) - No one ever got back to me on the details of why there was a huge increase in H2 2022 of placements who ghosted and were included based on the LinkedIn showing they had a job rather than responding to Codesmith directly. The CIRR specification doesn't have any guidelines around this and Codesmith hasn't responded to me with the process around it. >Part of the challenge is some of the major skeptics on our outcomes have their own coding programs and totally understandably want to report to their own standard and so raise questions about CIRR. What we don’t do is the standard approach of removing 40% of ‘people who weren’t job searching’ kind of thing or 1x 'highest offer'. - This is a passive aggressive statement about the bootcamp industry. Rithm and Galvanize have well documented, published standards they follow that aren't CIRR and they should be examined and compared to CIRR... implying CIRR is better is super arrogant. - I run an interview prep and mentorship program that doesn't publish many outcome because we have anyone paying from month to month memberships to unlimited memberships, most people are employed currently as very busy software engineers, no one does the same things or follows any fixed curriculum, people are full time to part time, junior engineers to principal engineers, and we don't feel like we can properly communicate outcomes in CIRR-like metrics because we're too small and too bespoke. - Seek to understand then be to be understood. >That’s on us to explain why we do it. I always thought we could just focus on the students, program, teaching etc but actually people reasonably want to understand how the outcomes are possible (esp when the CIRR report - as a ‘census’ requiring like 3x followups to every person to even be compliant - takes forever to produce and covers 2022-23) >So the other data that matters is the ‘snapshot’ - the latest outcomes (ie for April-May 2024) - \[[LINK](https://codesmithdocs.s3.us-west-1.amazonaws.com/Codesmith-Accepted-Offers-Latest.pdf?hsCtaTracking=5fa5766f-6dea-46ee-b25d-ecf7276ecee9%7Ceed80937-eb54-43e8-9542-eefb75bc9536)\] - 54 offers, median of $119k, highest offer in the $400k range. These obviously are only a snapshot and don’t give a rate of hired - because you need to survey all grads from a given time period - that’s the 2022-23 CIRR report that came out a few months ago. But it gives a window into latest results (so they’re down from 2022 high of \~$135k) - You shared some outcomes in November-ish from a two month window then, but how as December, January, February, and March and why aren't you sharing those too? You've shared a lot of data already so I'm fairly up to date with outcomes and those outcomes are not as strong. - **You have a lot of data on the H1 2023 grads and those CIRR worksheets are on going**. You also have been doing this for **10 years** and you have some insight into the trends. If you aware aware that H1 2023 grads have a **significantly lower** one year placement rate than your CIRR data (even if you haven't tallied all of them officially), I believe in my personal opinion that you have a duty to give an unofficial heads up about that. Again, I'm holding you to the bar of the best of the best. >But the outcomes are bigger than those first year offers covered by CIRR. Actually I gave a whole talk on the outcomes stats that matter to us - [LINK](https://docs.google.com/presentation/d/1FYbx8Cy4J6zfg96GE5aHNfGhDJe7sXcVttyYP5zUNDI/edit?usp=sharing) - of which CIRR ones are just a few  - I agree with this, a lot of things matter a lot more than outcomes >Things like promotion rate (100% between 5 and 7 years of graduation - double the rate of average in software engineering) and how many go on to start firms that use tech for relative good (not enough yet - we need to encourage this more) - This is absurd to me that you would mention this. You run a grad survey that is completely opt-in and only includes people who replied to it in the count. People who are struggling, who ghost or disengage aren't replying to your survey saying that they went back to their old job, or they had a really hard time and got mentorship from others. **If you are going to make passive aggressive statements about others following CIRR, don't publish this very weak data about promotions side by side with CIRR.** >And my fav ‘outcome’ of all - that over half of alums don’t use javascript/typescript (key language we teach) because they’ve become true software engineers. So that’s the other thing - it’s on us to center all those outcomes too, not just the first year salary and hired rate. It’s on the [new site](https://www.become-irreplaceable.dev/) so I’m happy about that - but more to do there.