Who You Calling a Jesse?

Understanding the metrics incubators or accelerators need to track

Like startups, incubators and accelerators are hard work but they aren’t startups — they are educational programs (and sometimes facilities) that use something resembling an apprenticeship model to move a (normally software) business through their life cycle at an accelerated pace. Incubators and/or accelerators are likely to offer some resources like a place to work and/or money as well but the value is in the educational program which should drive the metrics that are important to these organizations.

Success isn’t easy to agree upon with these programs and if you look at the math it isn’t a sure bet they are going to be making money. Measuring success in an education model isn’t easy either. Having what is essentially a marriage of a venture capital model with education does not make it easier. If you are running one the dozens of newly minted programs or are one of the thousands of hopefuls trying to get into the program, here are some things to keep in mind.

Baseline metrics

In my world there are only two styles of incubator/accelerators and that is TechStars or YCombinator which TechCrunch as tried to come up with numbers to compare:

• Total companies
• Total raised by companies
• Total rounds raised by companies
• Money raised after seed funding (money raised after the incubator process)

What TechStars tracks (and shares openly) beyond the above list:

• Number of employees
• Status: active/failed/acquired

Why do the numbers matter?

Paul Graham is open about YC’s numbers but he struggles with what success means and acknowledges it takes a long time to see the real success of YC.

Incubators and accelerators can and should make money, Betaworks demonstrated that by paying back its investors. Using the 316 number of companies that have been to YC and assuming that ‘rarely over $20 000 in investment‘ translates to:

• An average of around $18 000 or $5.7 Million has been invested by YC (apparently they might have invested more as they did follow on rounds when funding was scarce)
• The avg value of 210 of them is estimated to be $22.4 million each
• $627 million has been raised (according to techcrunch)
• $4.7 billion in total value of companies

In contrast, TechStars has 114 companies at:

• $18 000 each (plus now a $100k note option)
• $132.3 million has been raised by TechStars alumni.

Assuming that raised money accounts for at least 20-25% of shares in the companies (average), you can guess that the value of companies is not on par with YC companies but still impressive for only 114 companies.

I will throw a third into this mix, Mass challenge, who is clearly about education through apprenticeship. They focus so heavily on the programming side of things when they talk about:

• the number of applications, over 1000
• 111 startups, $103 million in funding raised within 12 months of completing the program
• 805 employees now at those 111 companies
• They don’t take equity

To me these numbers for Mass challenge demonstrate that their success is driving as many companies as they can through their program. The quality of the program is measured by the funding they get afterwards as well as the jobs created. TechStars measures that too.

A lot of these numbers being thrown around are used to inform entrepreneurs about the nature of the program, enticing them to be a part of it. Other metrics commonly shared:

• Number of mentors available
• Value of extra services like legal, accounting, software, etc

These are likely more vanity metrics than anything else.

Developing metrics for your incubator or accelerator to drive programming

When developing the programming you should design it around some basic questions:

• What is a success for a company or founder that enters the program?
• What is the time frame you expect them to realize that success?
• What are the activities you can measure now (eg. event attendance, company funding, jobs created, IP created, etc)?
• What should you measure later?
• Does volume of applicants, participants, funded companies, etc matter?

Then you need to build some instruments to measure, monitor, and adjust. Everything you do from speakers to mentors to other events should move the numbers one way or another for every cohort that is run through the program. There are no bad metrics to start, just start.

Building a funnel to help interpret the data

The higher the number of people at the top of the process, the more likely you will have great things coming out the other end. The more success you have at the bottom of the funnel will fill a growing pool of resources to feed back into the process. Where this starts for an incubator/accelerator is at the application level.

The basic funnel is illustrated above (yes it looks like Skok’s customer acquisition funnel). In each step there are a lot of actions that influence the outcome down the chain but to start:

1. Applications – this is your top of funnel where you collect information on people and make some decisions.
2. Interviews – at this point the teams get to meet you, the assessment should be working both ways.
3. Funded – companies that are accepted into the program.
4. Successful – those that meet your definition of success, at a baseline that should be all those that complete the program.
5. Alumni – this is the growing base on the other end of the funnel that you should keep engaged.

The additional definitions of success are more realistic. You will have companies that complete the program but fail to raise money or find revenue. However, you would still put that group into the alumni pool as they can help drive bigger success throughout the funnel.

The basic model is a science but building success is an art

If you look at the numbers of TechStars companies by location you see the results on funding post-TechStars vary a lot by cohort and location yet the model is ‘open’ and known. If running these programs was a science I would expect more consistant results.

The success metric for a VC focused incubator is profit but they need a secondary metric that is equally important — 500 Startups is to build a tight community of 500 startups, TechStars has its Alumni network that doesn’t reject stagnent or failed companies, YC has a very tight knit alumni group as well. All the other metrics  indicate the health of funnel is what I believe speaks to type of experience the entrepreneur is going to have being a part of it. 

Entrepreneurs should be asking the incubators why they are doing what they doing and be just as selective as the people on the other side of the table.

Early stage companies don’t need money, they need customers

The popular belief in Canada is that the tech startup world has been fairly light on investment dollars relative to other industries in Canada. Because there is such a disparity in seed or angel round investment size in Canada vs the US people tend to point to that as a reason people go south. The perceived result of the funding problem (and likely the weather) is that there are 350 000 Canadians in the Valley. No one can argue the talent to build global calibre tech companies exists in Canada (or at least has Canadian passports) but you can certainly argue Canada lacks that certain something to keep them here.

Five years ago Paul Graham observed that the total cost to get a tech startup started had dropped dramatically and will continue to do so.

So my first prediction about the future of web startups is pretty straightforward: there will be a lot of them. When starting a startup was expensive, you had to get the permission of investors to do it. Now the only threshold is courage. – Paul Graham, 2007

There is a lot of attention around getting young people money but does that help them? Does that keep them in Canada? I would argue that the ones that do need and can use capital don’t pull up stakes and leave town for the investment. They leave town (or the country) because they are missing something more valuable than money — customers, mentorship that helps them get customers, and a network of peers.

Know thy stage

The problem with comparing funding deal levels in Canada and the US is that it ignores the stage the company is in relative to the stage of US startups raising money for the first time. The Startup Genome report 01 and the Startup Genome Compass offers startups an excellent way to measure themselves against a benchmark of over 3 000 startups. In the report there is a table (shown below) that gives you some overall averages for all startups.

From the Startup Genome Report 01.

In last seven years of being involved in the Canadian startup community (mostly in Waterloo) and in the last three years leading what is arguably the best student focused incubator in Canada while founding my own startup. I saw dozens of companies peek into the Discovery phase, a few move on through to the Validation phase.

What I have seen happen before the discovery phase:

• Talk of raising money is used to pull in a large group of talent.
• Focus is not on customers, it is on technology or raising money.
• There is little help by way of mentorship that takes the time to understand the dynamic of the group.
• Mentors focus on finding a way to get them money so they can work full time.

Startups need to focus more on customer acquisition and growth in Canada, enough talk about raising money

There are so many business plan and pitch competitions one could make a career out of attending them. This gives a false sense of success because the ‘winner’ is determined on a lot of factors except their ability to actually get customers. The game becomes about (and has been it feels like) how to put together a report on an idea (business plan) and present in a way that makes you look confident.

The game is really about getting lots of people to give you their money because you provide value to them. What makes you better than others is that you are chasing a much bigger problem that will provide value to a full percentage of the world’s population. Bonus points if you change the world.

Employers, why asking for Facebook account access is bad

A recent flurry of articles have been published about employers asking for Facebook account access demonstrates the lack of understanding of social networks. Facebook is a personal social network that people use it as a primary communication tool for family and friends. It is also a trusted channel of sharing information — like email. The privacy settings in Facebook and all the issues around the Terms of Services speak to how people view it as a deeply private space. On top that, Facebook isn’t too happy about other people gaining access to your account.

When you ask for access to someone’s profile on Facebook I see the following issues:

• It contains information that is way outside of the scope of employment such as sexual orientation, religion, who they have a relationship with, who they are friends with, communications between friends and family, and more.
• People that have allowed friend access to that person have a trust relationship with that person — when a potential employer accesses that profile they assume the identity of that person in that relationship. (Update: I think this why Facebook is unhappy about this practice)
• The person that just gave you access to their Facebook profile isn’t concerned about the confidential information of others they just made available to a stranger. Does that say more about them than a picture of them slumped over with a beer in their hand from 5 years ago?
• People can (and do) set up fake profiles and carefully create the persona they think you want to see.

In the article published by AP, it seems to be perceived ok to ask for access or a friend request especially in public service jobs. What I really like is this part of the article when talking about what they might find on Facebook:

When asked what sort of material would jeopardize job prospects, Thomas said “it depends on the situation” but could include “inappropriate pictures or relationships with people who are underage, illegal behavior.”

Woah, wait a minute. What is inappropriate is completely subjective. Could it be simply a picture of a guy hugging another guy is inappropriate for that job? The answer that person would have might be no but once they gain access to someone’s profile how can they say it didn’t influence them? What other information do you get with that access? You gain access to all the information that opens you up to being accused of discriminatory practices. What would be considered discriminatory practices? From the US Equal Opportunity Commission:

• harassment on the basis of race, color, religion, sex, national origin, disability, genetic information, or age;
• retaliation against an individual for filing a charge of discrimination, participating in an investigation, or opposing discriminatory practices;
• employment decisions based on stereotypes or assumptions about the abilities, traits, or performance of individuals of a certain sex, race, age, religion, or ethnic group, or individuals with disabilities, or based on myths or assumptions about an individual’s genetic information; and
• denying employment opportunities to a person because of marriage to, or association with, an individual of a particular race, religion, national origin, or an individual with a disability. Title VII also prohibits discrimination because of participation in schools or places of worship associated with a particular racial, ethnic, or religious group.

Similar laws exist in Canada, United Kingdom, Australia, and throughout the European Union.

Just because the information exists now in social networks like Facebook should not mean that employers should gain access to that as a way of ensuring they make a good hiring decision. Furthermore, if you rely upon the social network information you could be fooled into thinking someone is a certain way because what they gave you access to was a carefully crafted fake profile to make them look awesome. I am not saying you shouldn’t use social networks in hiring – if you don’t currently use LinkedIn you are missing out on a very powerful tool for due diligence, in a professional way, on someone.

Update: asking for facebook access is illegal in Canada says lawyers

I just think employers should stay out of people’s Facebook profiles and mainly because they find out way too much information on someone and it may put the employer at risk.

Harnessing the power of peer pressure in the form of mentorship

When looking at mentorship or coaching in a community of startups or in a workplace setting I have often referred to the power of peer mentorship within a set group as a key component for developing talent. The number of times I have heard a founder say that “it was so nice to know other companies have the same problems as us” is high. The challenge with peer mentorship is that it seems that the education system is a bit conflicted about it. The lack of commitment to peer learning in education means that students need to learn about it on their own. I think is a blow to the development of an entrepreneur and/or a top tier employee. The benefits of peer networks wasn’t known to me until a few years into my working life and I think it put me a number of years behind in my professional development.

With some of the top talent in the tech world being drawn to the incubator/accelerator scene I think it is evidence that peer learning environments and communities are something that talented people are drawn to. TechStars or YCombinator are essentially peer driven learning programs.

Specific to startup talent development, how I think you can best harness peer influence working with early stage companies:

• Regular dinner events were the founders can demo what they are working on, network, and socialize in a comfortable setting – YCombinator does this as do most other accelerators, incubators. Founders and funders dinners achieve this as well.
• Open work space where people can openly share what is going with them right now.
• Someone keeping track of the progress and celebrating success.

There are tools like Yammer that enable employees to share and collaborate. I believe the internet has allowed people to build their own peer learning environment and that is having a big influence on the work place — I am just not sure most businesses understand that shift enough to use it to their advantage. The startup world certainly has.

Also, have a look at:

• white paper published recently by TribeHR on Peer Pressure in the workplace there is a ton of evidence cited as to why it is a phenomena you can’t ignore.
• Power of peer mentorship in college (slideshare)
• How to cultivate a peer coaching network (HBR)
• How sheep-like behavior breeds innovation in Silicon Valley

There are two types of successful startup incubators in the world: YCombinator or TechStars

Incubators and accelerators have but one purpose: move startups along in their life cycle at a faster pace than they would normally and increase the likelihood of a return by providing that service. If you are a startup looking at applying to an incubator you need to understand that the differences in how these programs differ go beyond the money they give you in exchange for equity.

An oversimplification of the incubator/accelerator space is to classify them as either a Y-Combinator (YC) or a TechStars (TS). If you really look at the booming world of incubators for high tech startups you see a model that either based on education and peers that is driven by a strong personality (YC) or a model that is more institutional, mentor driven, follows a script, and feels less personal but is more in line with how VC’s work (TS) (right in the middle is where I would place 500 Startups – which is arguably representative of a third type). There is plenty to be found about the differences but here is a bit of a deeper exploration into the differences.

Startup lifecycle

Startups have a number of key phases in development that is best outlined in Fred Destin’s presentation on startup lifecycle.

1. Start
2. Launch
3. Build
4. Chasm
5. Scale

With the 12-14 week cohort models, like YCombinator and TechStars, the focus should be on moving through starting and on to launch phase. There may be some that get into a build phase. The incubator or accelerator hopes that once they are done a 12-14 week program the startup will be in a much better position to move quickly through the build stage and at least take on the chasm phase.

Where I see the key difference between YC and TS is that YC seems to be able to get companies to go through stage 1 to 3 and they accept companies mainly in the start phase. TS seems to not attract a cluster of companies in a particular phase or not care about what phase a company is in.

The basics of an incubator/accelerator (whatever you want to call it)

Within the execution of any incubator or accelerator program there are, in my mind, 4 core stages in a typical cycle:

• Recruitment
• Onboarding
• In the program
• After the program

Within each of these of these stages there are a number of specific activities that all incubators do but in general they aren’t all that different.

Recruitment

YC currently leads the thought leadership with Hacker News, Paul Graham’s (PG) blog, and it’s success. Applicants fill out a form and once told they have an interview, travel to YC in Mountain View for an interview. They get just 15 min with a small panel and the panel does a bunch of tricks to the founders like carrying on side conversations – there are a lot of blog posts about that.

TechStars has adopted a more consistent process over it’s many affiliated programs (it appears) but they lack YC’s Hacker News or thought leadership (although they would claim otherwise). With Techstars there appears to be an affiliation with the Kauffman Foundation and the role they are taking in promoting the incubator model in general they have made themselves an authority in the space. From people I know that have been in the program it is a fairly standard process similar to raising Angel capital.

Onboarding

I am not sure on TS on-boarding but YC has a very short interview to decision to start of program window. YC has a little book that is like a long Wikipedia article written by Paul Graham that offers insights and baseline knowledge. From what I have been told the YC machine is pretty much immediately available to you when they say “you are in” — startups decide when to tell others. What is really interesting is that YC doesn’t announce it. They generally let a company know they are YC funded on the interview day but they don’t make a big announcement or anything.

Not having a big incubator announcement is a key difference here. I will assume that with TS it is just like YC in that they have decided to fund you, they are now available to you. However, TechStars (it appears) doesn’t approach announcing the cohort in the same way as YC — they announce them ahead of the program. For a startup this little difference could be a big one if you are concerned about managing expectations of outside investors as you go through the program.

In the program: peer mentorship, startup culture

Each program runs for roughly 3 months, 12-14 weeks, where mentorship, various events, and a demo day to close it off normally occur. Each week is important given that each team only has 3 months. Over three months there are phases you can generally identify:

• Teams becoming familiar with each other, their mentors, and what they need to do (first 2 weeks).
• The heads down getting stuff done phase (8-10 weeks).
• Funding mode going into Demo Day (2 weeks).

Other incubator programs are fairly similar with any given week involving office hours (optional or required) and a speaker/dinner. The office hours are used to check in and place goals on the teams. Throughout the term there are demo nights, which are used by YC as a way to put peer pressure on other teams that might not be moving as fast as others.

Where they differ here is in the education of the founder(s). From everyone I have talked to that has gone through YC it seems to me it is a very challenging but rewarding relationship for a certain type of founder. That would make sense as a certain personality type will work best with Paul Graham’s way of doing things and will excel. I am not entirely sure it is simply a hacker/coder persona as most assume. I think it is a personality and learning style that goes a bit deeper.

TechStars has a co-working model with parts very similar to YC. The key difference is that TS doesn’t have the Paul Graham approach to educating founders so you will get very different details depending on who is running the program. The character of the TS program can vary because it is so mentor driven and puts the onus on the startup to engage those mentors. There is a big plus to this approach as you are more likely to find a good fit in the large mentor pool for you and your company. TS also gives the startups a place to work where YC leaves them to find a house and work out of it.

After the program: Alumni network

The key value any incubator or accelerator provides after the program is the alumni network of companies that are now a few steps ahead (depending on the age of the incubator there could be alumni with very large companies) of the current cohort in the program. Over time these alumni are your best mentors and connectors.

It is at this phase where the greatest value is for the startup, I believe. You now have access to what the old folks call a big rolodex (social graph) that will open many doors which essentially leaves it up to the entrepreneur whether their company will succeed or not. There are few to no barriers, generally speaking.

Any alumni of YC or Techstars still have contact with the folks in their cohort and all cohorts along with Hacker News. Techstars Network is so big they have a conference just for alumni while YC taps its alumni for all kinds of things. Also, founders seems to find going through the program a second time is different but just as valuable. These massive networks of successful alumni with a flock of high profile admirers is very similar to that of Higher Education alumni networks, so much so it convinces me that this entire process is a form of higher education.

Programs that work copy YCombinator, even TechStars did

The current culture of education focused incubators started with YCombinator (started in 2005). I believe what we are seeing with the success of YC and TS is new take on graduate school. Both are different, both work, and people can have strong opinions either way. They feed a need that I don’t think people outside of incubators or startups fully understand yet, learning to be a founder is really hard. Being a successful founder is even harder. The bet is that if you help young founders focus on what is important they will see success earlier or just simply see what success looks like.

If you are looking at an incubator anywhere (there are lots of great programs out there) you need to understand that the money is secondary. You need to find a program that will fit with the way you learn and has companies that you want to work with. It is just like how you picked your University or College except this time it can cost you a lot more (in equity) if you are successful.