CIRR appears to be done and irrelevant now - Codesmith needs to get off the Titanic before it sinks (Personal Opinion)

CIRR appears to be done and irrelevant now - Codesmith needs to get off the Titanic before it sinks (Personal Opinion) As many are aware, [CIRR](https://cirr.org/) started out a business-league from Skills Fund to try to standardize bootcamp outcomes in the early days of bootcamps. While CIRR's stated goals were to create transparency in the Bootcamps industry, it was ultimately not a charity - and was a business league, like the Chamber of Commerce, whose practical value was promoting and marketing for it's member bootcamps (who pay fees to be members) that did particularly well. So as bootcamps started doing terribly - particularly in 2022 -> 2023, a lot of those backers left. **You can see this in how important "transparency" was when bootcamps were doing well, and how quickly and efficiently they posted outcomes, and how when outcomes are terrible everything comes to a halt - this is a result of CIRR being a business league protecting the interest of it's members as a priority over transparency.** CIRR is down to a handful of school members left. Codesmith is the only one left taking it seriously, because it's such a strong factor in their appeal to prospective students and numerous alumni have fought with me about how great CIRR is, one calling it "the gold standard". 5 months ago, when we were all eagerly awaiting H2 2022 outcomes - which never came and were the first sign of a decline, there was a surprise the last effort when a new Executive Director - Rachel - came in and [promised to reboot CIRR](https://www.reddit.com/r/codingbootcamp/comments/17bnwyb/council_on_integrity_in_results_reporting_cirr/), get more members, update the standards. She stated that H2 2022 outcomes would not be published because CIRR is going to start reporting on 12 month post-graduation timeframes instead of 6 month, so we'll have to wait six more months. Well it's been 6 months and this is the latest: 1. Rachel is no longer the President of CIRR and has moved on to a new full time job. I'm not sure who the new President is, but the person who promised all of the changes above failed to make progress and she moved on. 2. CIRR is now in a worse spot than it was before. They have removed all PAST outcomes from their website, so there is absolutely ZERO data posted there now and we can't even lookup old results. If their mission is transparency, they are going in the wrong direction here. 3. They claim there are "new standards" that compute outcomes using a more lenient 12 month window instead of a 6 month window they have used for the past 7+ years. We haven't seen those standards yet. In full transparency they should share those prior to new outcomes. Every open organization I know that has specs and standards publishes them for public comment and review far before they are implemented. Codesmith and CIRR, Codesmith is the only one left prominently supporting CIRR, so I will discuss them here: 1. Codesmith's link to CIRR data on their homepage has been a dead-link 404 for months now 2. In early February, Codesmith promised CIRR outcomes in "a couple of weeks" and it's been over a month now, and there are no outcomes. 3. Codesmith's representative at CIRR no longer works full time at Codesmith and has become an advisor as of March 2024. 4. The previous executive director said that Codesmith was free to publish their own data for H2 2022 SIX MONTHS AGO, and they elected not to, and they continue to show H1 2022 data in their info sessions and on their website. Since CIRR doesn't actually have any official data on their website anymore, why not show people H2 2022, or the new FY 2022 outcomes? **This is Codesmith playing marketing games to delay very poor 2022 data as long as possible.** 5. Codesmith has published some salary data on their [blog](https://www.codesmith.io/blog/early-look-2023-outcomes-and-analysis) that show offers in 2023 are way down compensation-wise. They didn't mention placement rates at all, which is simply covering up the fact that placement rates are much lower for 2022 grads. 6. Codesmith is in a really hard spot because a large number of alumni say that they found Codesmith via it's CIRR outcomes. But CIRR is the Titanic and it's sinking, you can sink with it or you can move on and build a new ship. 7. If Codesmith keeps telling people to go to CIRR and promoting CIRR and people go to CIRR's website and there is NO DATA, that will make them more suspicious and not want join. It honestly looks like a scam (in my complete personal opinion as a user). I saw a post from an Alumni touting that Codesmith has been in CIRR for 7 years and how important CIRR is and go to their website to see - and the website has NO DATA AT ALL.... it's looks suspicious! Summary: I'm calling all this out because outcomes in the bootcamp industry all over the place, from making your own standards, to publishing carefully selected numbers w/ fine print, to promoting CIRR, to just not publishing any data at all. **At the end of the day it's all marketing, CIRR or no CIRR. Evaluate the market carefully and choose the right program for you - and choosing no program is an option too.** ​ ​

Just to be clear, this post was a personal opinion and not one of Formation and not about Formation. RE: Formation, I take that as feedback for sure, I don't think the 750K number is really effective too, but it's accurate and it's what our competitors do, so we took the easy there as we focused on other parts of our website.... Which I think is the fundamental reason we're on such different pages here, Formation isn't a bootcamp and CIRR-type data makes no sense. We've tried to cut numbers with artificial time windows for banks and loan providers and all those people love them and support us strongly, but we don't share them publicly because they used for math for finance people and don't help an individual understand their journey and likely would only mislead them because of the unique commitment and path that each person takes.... the timeframe is largely up to the person. Not all of our placements are public, but there are a growing number of people who have been with us for 1+ years who intentionally paused or slowed their job hunt process in 2023 because of the market expecting to ramp up when the market bounced back. We kept out promise of supporting them indefinitely until they got a job, and a bunch are getting jobs, and giving us fantastic internal ratings in their exit surveys. I expect AND STRONGLY HOPE that we see way MORE people with us over a year getting jobs! The key thing here is feedback. We change things every day and we listen. I've sent your feedback to our marketing team. We've been focusing more senior and have helped about a dozen people get jobs at Meta in the past few months in the E4, E5, and E6 level of experience, and we will continue to change and adapt in real time to the market and all we care about is people getting great outcomes and new jobs they love.

I wouldn't say it's the "only objective metric" no. It's a reliable and good signal to use, combined with other data and information. The outcomes themselves are more indicative of the job market than the programs themselves, so judging a program by it's outcomes alone was never a good idea, even in the good times. The fact that a CIRR report exists checks off one box of legitimacy, but it's far more concerning that Codesmith and possibly others had H2 2022 outcomes audited and ready for 6 months and won't share them, while they tout H1 2022 numbers in marketing. Even if this was not CIRR data, if that fact was true I would be demanding transparent placement rates before signing an agreement if it was me, but others might have a different bar.

That's why I shared this! CIRR is good but that conclusion is not correct and hence why I'm even posting this

I wish I had a strong answer, but I don't. I think better information would include: - backgrounds and experience of people before entering - more clarity on the types of jobs people get - more satisfaction related qualitative info, i.e. "how much do you credit your bootcamp in getting the job you got" - histograms on placement times and salaries instead of medians and averages - including all forms of comp in data, like bonuses, benefits, etc... - salary increases - i.e. someone leaked data that mid last year the average placed student at Codesmith had a current/previous salary of about 70K - which is higher than the median placed salary at some bootcamps - more transparency on hiring companies. too many bootcamps show a wall of logos of amazing tech companies, and talk about anecdotal placements there, but what are all of them. e.g. are people making $120K at a design agency vs $100K as a Pinterest SWE apprentice.

They ask when you get an offer and fill out a form, if you put in 0 or blank then they wouldn't know and that sheet shared was the average of people who shared. I'll try to dig up the post, so this is just illustrative and not fact until I find it.

What information have I refused to release? I've explained extremely transparently what information we have and don't have, and why we do what we do. You can see an entire list of every single placement's company on our blog. If you think that CIRR would work for Formation, you don't understand what Formation is and how it works, and coordinated downvoting my comments doesn't make you all understand what Formation is better than I do. If you want me to give you a CIRR report you have absolutely no idea how Formation works and you should not be signing up whatsoever until you understand what it is and what you are getting. We very transparently explain the average compensation gain of a placement which tanked last year from $100K to $80K and will hopefully be much higher in 2024. We were also transparent about how top tier placements tanked from 75% of outcomes to 50% (which is back up to 75% for 2024 so far!). And we explain exactly how we calculate it in a way a scientist could reproduce because that works for a junior engineering making 80K who wants a $160K job and for a senior engineering making $220K who wants a $500K job. So that's what we explain. We can and want to share WAY MORE so I accept the criticism, but saying that I'm not transparent is false.

Yeah I have screenshots and records for private use but I can't find the original post quickly. They do ask now though yeah.

Which competitors am I discrediting? Our competitors are Interview Kickstart, Pathrise, and Outco (before they kind of went MIA) and I never say anything to discredit them on Reddit or anywhere and I've even ENCOURAGED people to go TO SPECIFIC ONES in specific situations, e.g. for Product Management - which we don't help weith. Our recruiters talk day in and day out with people considering between these options (or only considering Formation) and these are our competitors. It comes up like ONCE a month that someone is considering a bootcamp OR Formation and the recruiters escalate to see if the person is experienced enough for Formation. The bootcamps vary from Codesmith to Springboard and the majority of the time if the people don't have experience we tell them to go to a bootcamp. If someone has legit SWE experience for 1+ years they should not go to a bootcamp in almost all circumstances but if for some reason a bootcamp was best for a specific person I would tell them and my team would tell them. It's far far far more common that bootcamp alumni (specifically Codesmith, Hack Reactor, Rithm, Launch School) come to us 1 to 5 years LATER ON for future job transitions.

We aren't a school, don't teach anything, and don't educate. What we do is unique and the closest competitors are IK and Pathrise and we're still unique amongst then. It's on us to explain what we do in our marketing and materials, but we haven't figured it out yet so I'm here telling you directly from the source of truth to try to help. I would love if you actually listen and ask questions to clarify and talk to me about what Formation does and then I don't have to write essays.

The problem I have with Codesmith alumni is that I'm not in fact attacking Codesmith or defaming them. I'm reporting on the facts, some are good and some are bad. But the WORST of all is the market for entry level jobs, which Codesmith has absolutely no control over, and that's the primarily reason people aren't going to bootcamps right now, not anything that Codesmith is doing poorly. Formation isn't a competitor to Codesmith but I'm happy to tell you how we are doing. You can read it on our blog. Our 2023 offers tanked and people's average first year total comp increase dropped from $100K to $80K. Our top tier placements tanked from 75% of all placements to 50%. This is on our blog since December 2023. In 2024, which is not published anywhere, top tier placements are back up to 75% and average comp increase is $117K. We don't have any placements rates because it doesn't make sense the way Formation works. We've had hardly any withdrawals prior to placement in 2024. On the downside, a good number of people in 2023 delayed or paused job hunting for the market to resume, so there are a growing number of people who have been training with us for more than a year. That costs us a lot of money and we're sticking to our promise to support them and we have enough money from VC funding to do so. One of these people received a record all time high offer, so I'm optimistically hoping that as these senior people resume their job hunt they will find good jobs and I'm will go to the end of the earth to help them. I don't know anyone in the past year who has been choosing between Formation and Codesmith. Serious question, not rhetorical: what incentive do I have if I were just bashing Codesmith? Think about it rationally, what incentive? We have two people I know of that chose Formation over Codesmith 2 years ago. We have a few dozens Codesmith alumni who have come to Formation for their 2nd, 3rd, 4th job searches and some immediately post Codesmith as well. So what incentive do I have to "defame" a good source of people to my business? You sound like more of a Codesmith supporter, but a lot of people aren't and then send me a lot of interesting documents and screenshots, a lot of people feel like their leaders just defend themselves instead of listening and even some super close long time supporters have had enough. I don't really want to be in this position, but I can't send people there knowing these things. No business is perfect, Formation certainly is far from perfect too, we all want to become better and better every day.

For the past year we don't accept anyone without a year of SWE experience, and literally a handful of people who appeal to come in with say 6 months of experience but are clear on their goals and aligned, so no one recently does them back to back. We've had way more people come back to Formation twice and pay us twice (or three times) than we have people who have done Formation immediately after graduating Codesmith. The fact that people come back to Formation multiple times and pay us each time is a very clear indication we are not remotely anything similar to Codesmith. We don't teach anything. The personal trainer analogy is much stronger. Hire a personal trainer, get into shape, good for a few years, have new training goal, get into better shape again, good for a few more years. Have a kid, need to make routine changes, get into shape again. \`I think it is very difficult to objectively communicate its value\` - EXACTLY MY POINT! That's why we don't publish bootcamp CIRR reports. The value people get is hard to put into a number and one of the strongest signals is people COMING BACK AGAIN and giving us industry high NPS scores (i.e. they recommend us to their friends at a very high rate). The gain in first year total TC is the most normalized number we have so far to communicate value. How do we prove it works? BENCHMARKING. We conclusively prove people's skill level has increased and you see that as a Fellow on a weekly basis. It doesn't work for everyone, but it works for most people and our job is to only let in the people it will work for. But again, this isn't a CIRR report and it's not dollar value, it's skill improvements and there's no standard for that, closest thing is the GCA and it's iffy. P.S. People don't get $500K offers without having 6+ YOE and being at the canonical FAANG-Senior E5/L5 level correct.