Council on Integrity in Results Reporting (CIRR)

Hi Rachel, I have a bunch of questions! I very openly have numerous criticisms of the CIRR standards but I also appreciate the vision of having standardized outcomes. My goal is to ask fair questions with no shades of opinions and hope to have constructive discourse. 1. Why are the results delayed for Q2 2022 compared to the Q2 2021 timeframe for release? 2. Why did the domain name expire earlier this year and no one renewed it for about 10 days afterwards during the domain recovery period? 3. What are the annual fees for being a "member school"? 4. Why is CIRR a 501c6 business league instead of a 501c3 non-profit? 5. What's the process for making changes to the standards and how to members contribute to that? (this might be a very long answer but just a high level overview) 6. What is the role of a board member of CIRR and what does the board do? 7. Why are outcomes reported as percentages instead of both percentages and absolute numbers? 8. If a member school doesn't follow specific CIRR standards because they don't feel they apply to their school, why does CIRR allow their results to be posted without a disclaimer? (This is reference to Codesmith's treatment of their TAs, whereas other schools report TAs as "hired by school", Codesmith openly considers TAs students and changes their graduation dates, even though it's a paid role) I might update more questions, I wasn't expecting this post and timebox my Reddit comment time!

Thanks! Again, very open to debate, but every bootcamp I've talked to anecdotally has struggled this year with enrollment and has noted longer placement times and lower salaries on average. Since the market is improving at least temporarily, I think delaying the results from the tough market until January could be harmful to students who want to know what's been going on recently and make more informed decisions about the timing of going to a bootcamp.

I know AMAs are a lot so I'm going to hold off commenting until a bit later and there's some breathing room but I wanted to comment on just this timeframe change right away so it gets read by more people: * Opinion: from my anecdotal conversations with bootcamp leaders, the H2 2022 and H1 2023 outcomes are significantly lower than in the past. I'm concerned delaying the timeframes will cover up the bad market in 2022 instead of acknowledging it transparently so that people can make informed decisions about such a bit live decision of changing careers. * Opinion: in anticipation of increasing the "hired by" range to 12 months from 6 months. Most bootcamps are 10 to 20 weeks long, which is about 2 to 4 months. If someone is changing their careers and going all in on a full time bootcamp for 2 to 4 months, and it takes an additional 12 months to get a job, there is A LOT that happens in that 12 months compared to the 2-4 months at the bootcamp. People might write way more code, do volunteer projects, do more courses or OTHER PROGRAMS in that time and all kinds of things within a whole year that could contribute to the person's outcome beyond the initial bootcamp. The bootcamp is the "seed" that kicked this off, but they shouldn't get credit for everything that happens in the 12 months after. The 3 and 6 month timeframes help capture the direct impact the bootcamp had on the job. * I'm bias here because my company works with a lot of people post bootcamp, and we see a handful of people who go to a bootcamp -> get a contract for 6 months -> come to us -> get a great job, all within 12 months of graduating from their bootcamp, and really the bootcamp played some but not all of a part in getting that permanent job.

\+1 to showing more data, and having like 4 time periods. Opinion: In the "what you can say in marketing" clauses, I think it should stay at the six month as this "transition" happens. My fear would be bootcamp websites showing side by side 2021, 2022, numbers and the 2022 look the same but are 12 month windows and the 2021s are 6 month. Based on the info sessions I've been to and marketing I've seen, I would expect companies (which is totally what they should do) to present the best numbers they are allowed to present. Opinion: In higher education people spend 4 years doing a degree and then spending 1 year to get a job is much different then people spending 2-3 months to get a certificate and spending 1 year to get a job.

Follow up question: is any specific member school contributing additional funding to support these changes or new initiatives? Based on the member fees specified and number of members, this seems like an ambitious expansion cost-wise.

This was answered above. They are changing the rules with the 6 month time period to a longer time period because placements are way down in this market, so there won't be any reports until January

I added a correction to my comment above. So Codesmith has been telling prospective students and staff their CIRR report will come out any day now. Given that this is changing and it will now come out in January is there anything stopping them from self publishing an official report themselves without the CIRR designation?

Does that mean that the expectation is that if someone wasn't following CIRR standards, they will have to pass a consistent audit in order to be posted? FOR REFERENCE OF WHAT I'M TALKING ABOUT: I mean they published [this](https://www.codesmith.io/blog/codesmiths-commitment-to-cirr-transparent-reporting-frequently-asked-questions) and stated: >Who is included in the “hired by school” statistic? Codesmith deeply values team members who grow their careers from the ground up and have a first-hand understanding of the mission behind the work that we do. For this reason, we appreciate instructors who have completed the immersive program themselves and understand the student experience. When you see the percentage hired by school on a Codesmith CIRR report, these are a very small handful of graduates that have moved on to work full time as instructors or Software Engineers for the company. N**ote that fellows are not included in this portion of the report (this is a standard that Codesmith has set within our own reports, and is not a governing standard within CIRR as an organization).** This percentage can vary from 1.5 to up to 5% based on our academic team hiring needs. How are fellows included in the report? Codesmith fellows are graduates of the program that stay on to assist the team in core mentorship responsibilities for incoming students, similar to a TA in a traditional university. Fellows are considered to “graduate” the program once they finish their fellowship, typically after an additional 3 months (though some fellows do extend their contract and stay on longer, and a small handful grow to have long-term careers at Codesmith). Despite working on a 3-month contract after graduation, most fellows obtain roles shortly after finishing their contracted positions, and for that reason are included in the report with their original student cohort. I think the CIRR standard is clear that since this is a paid contract role that it would be a placement, and should be in the hired by school bucket. And additionally that these people met all of the publicly stated graduation requirements prior to the fellow job and should be considered graduates on their real graduation date. If a student choose to take a $50K job as a fellow, that's their choice, just like they can choose to pursue a $120K job as a SWE. The purpose of these terms was to prevent bootcamps from 'juicing their placement rates' by hiring back students. So any other school doing this would include those people in their stats and acknowledge that some of the placements were hired back by the school transparently. In Codesmith's excluding them 'juices their already strong numbers' in the other direction, by excluding the low salaries of fellows that bring down their averages. If fellows get another job still within six months of their original graduation date - as Codesmith says "most" do, they would use the higher paying job, just like CIRR allows them too. I don't really care either way, but someone reading the report by the letter would not know about this because it's a very long blog post, and if the largest member of CIRR makes up their own rules, I don't think that sets a good example for other members.

That all sounds great, let me know if I can help!

This also sounds wonderful!

In all fairness, Codesmith literally says in their info sessions "If you are admitted into Codesmith you are already employable as a junior engineer and Codesmith will turn you into a mid-level or senior engineer". If you know anyone there you yell at them with the same insults and tone as your comments here.