Line by Line Rebuttal to Codesmith CEO dodging question about placement rates in a challenging market

Line by Line Rebuttal to Codesmith CEO dodging question about placement rates in a challenging market **DISCLAIMER: these views are my personal opinions as I see them and they don't represent anyone but me.** u/WillSen If you call yourself the best of the best, you need to hold yourself to that bar and respect others who are holding you to that bar too by responding with facts and arguments to every challenge rather than ban people who point out things you don't want to answer. I'm unable to reply in the Codesmith subreddit because I'm permanently banned. Anyways, someone asked the Codesmith CEO in an AMA today [link](https://www.reddit.com/r/codesmith/comments/1dofj3a/comment/la9fv9w/) >There has been a large share of skepticism towards the results that Codesmith claims to produce with job acquisition rates, salaries, etc. since the company does not share its raw data, e.g., claiming that 90% are hired after 6 months but not showing the raw data for how the 90% number is collected (the 90 number is arbitrary in this example). >When I have personally inquired during my tenure, I was either ghosted by Codesmith staff or rudely rejected. Can you speak as to why Codesmith has chosen this method of hiding the data? I'm dissecting the [response](https://www.reddit.com/r/codesmith/comments/1dofj3a/comment/la9xs01/) : >We report to CIRR (Council for integrity of results reporting). It gets a LOT of attention but I think it’s good to hold CIRR to such a high standard because people do take it so seriously (huge number of applicants say the reason they know about Codesmith is CIRR - it’s not like we ever advertise - although we are finally doing some ads now) - This is why Codesmith is defending CIRR so strongly and keeping it alive as one of three remaining schools. **IT IS MARKETING FOR THEM and at least they are being transparent about that.** I don't have any problem with CIRR, but it needs to be critically examined as a marketing tool and understood as a piece of the puzzle, not something applicants make their entire decision about. >Ultimately what we have to do and haven’t always done well is explain the how and why of the outcomes. It makes no sense for a random coding bootcamp (codesmith) to have had \~$135k median salary and 80 or 90% hired rate in 2022 (now btw $120k and \~80% hired rate in the last census 2022-23). So people reasonably look at the data with a close eye >All we can do is follow a shared standard [https://www.cirr.org/standards](https://www.cirr.org/standards) that is comprehensive (includes every single student) and transparent \[[worksheet](https://docs.google.com/spreadsheets/d/197WkH1sXYJwvtqwG2EontX3pxzNr8cRSzhqRLDKARaI/edit?gid=0#gid=0)\] and then have it audited. We even got an audit firm ([White & Co](https://www.codesmith.io/blog/codesmith-outcomes-reporting-a-conversation-with-james-white-of-banks-finley-white-company)) that are themselves audited (by AICPA - the accounting industry’s own auditors) - No one ever got back to me on the details of why there was a huge increase in H2 2022 of placements who ghosted and were included based on the LinkedIn showing they had a job rather than responding to Codesmith directly. The CIRR specification doesn't have any guidelines around this and Codesmith hasn't responded to me with the process around it. >Part of the challenge is some of the major skeptics on our outcomes have their own coding programs and totally understandably want to report to their own standard and so raise questions about CIRR. What we don’t do is the standard approach of removing 40% of ‘people who weren’t job searching’ kind of thing or 1x 'highest offer'. - This is a passive aggressive statement about the bootcamp industry. Rithm and Galvanize have well documented, published standards they follow that aren't CIRR and they should be examined and compared to CIRR... implying CIRR is better is super arrogant. - I run an interview prep and mentorship program that doesn't publish many outcome because we have anyone paying from month to month memberships to unlimited memberships, most people are employed currently as very busy software engineers, no one does the same things or follows any fixed curriculum, people are full time to part time, junior engineers to principal engineers, and we don't feel like we can properly communicate outcomes in CIRR-like metrics because we're too small and too bespoke. - Seek to understand then be to be understood. >That’s on us to explain why we do it. I always thought we could just focus on the students, program, teaching etc but actually people reasonably want to understand how the outcomes are possible (esp when the CIRR report - as a ‘census’ requiring like 3x followups to every person to even be compliant - takes forever to produce and covers 2022-23) >So the other data that matters is the ‘snapshot’ - the latest outcomes (ie for April-May 2024) - \[[LINK](https://codesmithdocs.s3.us-west-1.amazonaws.com/Codesmith-Accepted-Offers-Latest.pdf?hsCtaTracking=5fa5766f-6dea-46ee-b25d-ecf7276ecee9%7Ceed80937-eb54-43e8-9542-eefb75bc9536)\] - 54 offers, median of $119k, highest offer in the $400k range. These obviously are only a snapshot and don’t give a rate of hired - because you need to survey all grads from a given time period - that’s the 2022-23 CIRR report that came out a few months ago. But it gives a window into latest results (so they’re down from 2022 high of \~$135k) - You shared some outcomes in November-ish from a two month window then, but how as December, January, February, and March and why aren't you sharing those too? You've shared a lot of data already so I'm fairly up to date with outcomes and those outcomes are not as strong. - **You have a lot of data on the H1 2023 grads and those CIRR worksheets are on going**. You also have been doing this for **10 years** and you have some insight into the trends. If you aware aware that H1 2023 grads have a **significantly lower** one year placement rate than your CIRR data (even if you haven't tallied all of them officially), I believe in my personal opinion that you have a duty to give an unofficial heads up about that. Again, I'm holding you to the bar of the best of the best. >But the outcomes are bigger than those first year offers covered by CIRR. Actually I gave a whole talk on the outcomes stats that matter to us - [LINK](https://docs.google.com/presentation/d/1FYbx8Cy4J6zfg96GE5aHNfGhDJe7sXcVttyYP5zUNDI/edit?usp=sharing) - of which CIRR ones are just a few  - I agree with this, a lot of things matter a lot more than outcomes >Things like promotion rate (100% between 5 and 7 years of graduation - double the rate of average in software engineering) and how many go on to start firms that use tech for relative good (not enough yet - we need to encourage this more) - This is absurd to me that you would mention this. You run a grad survey that is completely opt-in and only includes people who replied to it in the count. People who are struggling, who ghost or disengage aren't replying to your survey saying that they went back to their old job, or they had a really hard time and got mentorship from others. **If you are going to make passive aggressive statements about others following CIRR, don't publish this very weak data about promotions side by side with CIRR.** >And my fav ‘outcome’ of all - that over half of alums don’t use javascript/typescript (key language we teach) because they’ve become true software engineers. So that’s the other thing - it’s on us to center all those outcomes too, not just the first year salary and hired rate. It’s on the [new site](https://www.become-irreplaceable.dev/) so I’m happy about that - but more to do there.

So Codesmith genuinely has good intentions, and they do some things incredibly well. They do a lot of things not very well. That's reasonable, no company is perfect. I believe in every number Codesmith presents is trying to be accurate, while also being marketing and that is maybe a similarity to Lambda School. Austen presenting what he felt was accurate information spun in very interesting ways - ex. 100% of cohort placed with very small sample size (not revealing sample size of 1) Ultimately it comes down to outcomes. If you have good material to work with, and spin the marketing positively, then you have success. If you don't have good outcomes and spin the marketing, you end up potentially with problems and people being mislead. Codesmith continues to have good outcomes relative to it's peers in the bootcamp industry, however the elephant in the room is that the INDUSTRY is doing very poorly. Codesmith is pivoting to a narrative about the "modern software engineer", which is about justifying people taking non software engineering roles that combining their past experience with coding. I LOVE THIS IDEA! But I HATE that they frame this as THE "modern engineer" across the whole industry because it's just not true. We're about to see an exploding in technical jobs created by AI, and we're going to need all of these people to fill those roles. Instead Codesmith is clutching to their pride about creating the 'software engineering leaders of the industry' instead of engaging with people like me that can help. I got yelled at for 3 minutes straight by the CEO in a large call recently, for example instead of welcoming my good intentioned challenges.

Yeah and my monitoring and tools are showing manipulation. A Codesmith employee was given a warning because of that and all of their accounts are at risk of being banned.

Imagine I was an anonymous account that posted this. If you understand Formation really well, please suggest the outcomes we should be presenting and how we should handle the situations I explained above, as I explained above, we haven't figured it out it yet and that's why I'm open to engaging with people to talk about it. Instead of Will's attitude of ban and ignore. He hasn't ONCE contacted me directly to explain ANYTHING. Downvoting without understanding doesn't make people right.

It's mostly alumni and not employees, there aren't many people left and they are busy some I've talked to are working on new jobs and not 100% drinking the koolaid.

I can't speak for people, the vast majority start of with: 'I'm a codesmith grad/alumni/etc... and I really appreciate you presenting things as they are. I don't agree with some of the advice and I'm majorly struggling and I feel like Codesmith isn't helping if I don't follow the norms, can you help' And then I try genuinely hard to advise and help the people and get into conversations.

I have conversations with a number of bootcamp staff and leaders from many places because I have a presence in the industry. Both as a leader of this subreddit, as one of the former top engineers at Meta, as someone who works with bootcamp grads from all the programs later on in their careers. If someone was talking about me or my company, I would have engaged them on the first message to talk about it and understand each other's point of view. If the person refused to talk to me and kept going, then I would call them out and ask why. Build bridges not walls.

Here are the most recent placement submission, in order unedited that I grabbed, and redacted tiny companies. Our outcomes are incredibly strong. The time it takes to get them is very long. Meta \[REDACTED STARTUP\] Amazon Meta Atlassian Paylocity Disney \[REDACTED STARTUP\] Netflix Gusto Amazon Western Union Meta Microsoft Microsoft Willow \[REDACTED STARTUP\] Reddit \[REDACTED STARTUP\] Microsoft Strider Technologies NVIDIA Meta

There's a difference between saying "my opinion is that the numbers on your website are deceptive" and "your numbers are deceptive" unless a person currently contains evidence that the numbers are illegally deceptive then one of these statements is an opinion that is totally fine and the other is libel. Those comments were deleted because the person is personally attacking in the comments and was previously warned. In my opinion, it's the equivalent of misgendering someone intentionally or calling someone a nickname they asked you to not call them. I flagged this to the other moderators in case they have different opinions and want to allow them.

No one should be choosing Formation over a bootcamp at all. a We work with people to support their goals. Right now we have observed very clearly that there is no systematic way for people with no experience to get jobs and we therefore don't take new people with no experience. We still have people with us with under a year of experience for 1.5ish years because of our indefinite support promise. We had three ex Facebook senior engineers start recently, current Google Manager, like these are not people considering Codesmith, Hack Reactor, Springboard, etc... Or are you saying they are? That a staff engineer at Meta is choosing between Codesmith and Formation? They are choosing between Interview Kickstart and Formation. I don't really know how to explain this more clearly, but I'll keep trying because I don't want anyone coming to us for the wrong thing, it's a waste of everyone's time.

I can connect you with some of the people to find out. $2.5K a month is by far the most expensive option and most people choose a package (you can see more options in post-application stage) but the pricing is meant to be a range of pricing for a very very wide range of circumstances. If you are time sensitive you are looking at different options than if you want unlimited support. And if you want unlimited support you can balance between paying all upfront and paying half based on how much we increase your salary. We're far from perfect but we're working hard to support engineers and we feel we have a good product and experience and we make hundreds of changes every week to adjust to the market and to improve the experience based on feedback. The main reason though is if you are a senior engineer and super busy, you can just dial into Formation based on your schedule dynamically every week, adjusting to your busy real life, and not think too much - just show up and practice and we move you along based on our benchmarking and your feedback. Some of these people wouldn't consider a job transition with out this kind of system. Another odd thing is that people have come back to Formation a number of times and happily paid each time. So we're not selling you secrets, we're selling you support, guidance, efficiency, negotiation, feedback, benchmarking, and practice.

Common: no, most engineers went to a top CS school They then looked at the best engineers at Meta, looked at the schools they went to, and then recruited from those schools. BUT, a number of "best of the best" engineers either didn't have a CS degree or went to a not well known school at all. The theory was the best of the best will find Meta or Meta will find them through acquisitions or because they like hacked into Meta and were offered a job. The best way to get in right now without a top CS degree is to get another job for 2 years and then apply for the Rotational Engineer program. At Formation, we've sent maybe 10+ people through there and it's super super ideal for non traditional grads with 2+ YOE at non-top tier companies.