In this project, we’ll aim to find the two best markets to advertise our product in — we’re working for an e-learning company that offers courses on programming. Most of our courses are on web and mobile development, but we also cover many other domains, like data science, game development, etc.

Understanding the Data

To avoid spending money on organizing a survey, we’ll first try to make use of existing data to determine whether we can reach any reliable result.

One good candidate is the data from freeCodeCamp’s 2017 New Coder Survey. freeCodeCamp is a free e-learning platform that offers courses on web development. Because they run a popular Medium publication (over 400,000 followers), their survey attracted new coders with varying interests (not only web development), which is ideal for the purpose of our analysis.

The survey data is publicly available in this GitHub repository.

# Load the required libraries

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
pd.options.display.max_columns = 150

link = 'https://raw.githubusercontent.com/freeCodeCamp/2017-new-coder-survey/master/clean-data/2017-fCC-New-Coders-Survey-Data.csv'
survey = pd.read_csv(link, low_memory = 0)
print(survey.shape)
survey.head()

(18175, 136)

Checking for Sample Representativity

As we mentioned earlier, most of our courses are on web and mobile development, but we also cover many other domains, like data science, game development, etc. For the purpose of our analysis, we want to answer questions about a population of new coders that are interested in the subjects we teach. We’d like to know:

• Where are these new coders located.
• What locations have the greatest densities of new coders.
• How much money they’re willing to spend on learning.

So we first need to clarify whether the data set has the right categories of people for our purpose. The JobRoleInterest column describes for every participant the role(s) they’d be interested in working in. If a participant is interested in working in a certain domain, it means that they’re also interested in learning about that domain. So let’s take a look at the frequency distribution table of this column and determine whether the data we have is relevant.

# Frequency distribution of 'JobRoleInterest'

survey['JobRoleInterest'].value_counts(normalize = True) * 100

Full-Stack Web Developer                                                                                                                                                        11.770595
  Front-End Web Developer                                                                                                                                                        6.435927
  Data Scientist                                                                                                                                                                 2.173913
Back-End Web Developer                                                                                                                                                           2.030892
  Mobile Developer                                                                                                                                                               1.673341
Game Developer                                                                                                                                                                   1.630435
Information Security                                                                                                                                                             1.315789
Full-Stack Web Developer,   Front-End Web Developer                                                                                                                              0.915332
  Front-End Web Developer, Full-Stack Web Developer                                                                                                                              0.800915
  Product Manager                                                                                                                                                                0.786613
Data Engineer                                                                                                                                                                    0.758009
  User Experience Designer                                                                                                                                                       0.743707
  User Experience Designer,   Front-End Web Developer                                                                                                                            0.614989
  Front-End Web Developer, Back-End Web Developer, Full-Stack Web Developer                                                                                                      0.557780
Back-End Web Developer,   Front-End Web Developer, Full-Stack Web Developer                                                                                                      0.514874
  DevOps / SysAdmin                                                                                                                                                              0.514874
Back-End Web Developer, Full-Stack Web Developer,   Front-End Web Developer                                                                                                      0.514874
Full-Stack Web Developer,   Front-End Web Developer, Back-End Web Developer                                                                                                      0.443364
  Front-End Web Developer, Full-Stack Web Developer, Back-End Web Developer                                                                                                      0.429062
Full-Stack Web Developer,   Mobile Developer                                                                                                                                     0.414760
  Front-End Web Developer,   User Experience Designer                                                                                                                            0.414760
Back-End Web Developer, Full-Stack Web Developer                                                                                                                                 0.386156
Full-Stack Web Developer, Back-End Web Developer                                                                                                                                 0.371854
Back-End Web Developer,   Front-End Web Developer                                                                                                                                0.286041
Full-Stack Web Developer, Back-End Web Developer,   Front-End Web Developer                                                                                                      0.271739
Data Engineer,   Data Scientist                                                                                                                                                  0.271739
  Front-End Web Developer,   Mobile Developer                                                                                                                                    0.257437
Full-Stack Web Developer,   Data Scientist                                                                                                                                       0.243135
  Mobile Developer, Game Developer                                                                                                                                               0.228833
  Data Scientist, Data Engineer                                                                                                                                                  0.228833
                                                                                                                                                                                  ...    
  Product Manager,   Mobile Developer, Data Engineer, Back-End Web Developer,   Data Scientist                                                                                   0.014302
  Product Manager,   Data Scientist, Data Engineer                                                                                                                               0.014302
Data Engineer, Full-Stack Web Developer,   Front-End Web Developer,   DevOps / SysAdmin                                                                                          0.014302
Game Developer,   Front-End Web Developer, Back-End Web Developer, Full-Stack Web Developer,   Mobile Developer, Information Security,   Data Scientist                          0.014302
  Product Manager,   User Experience Designer,   Front-End Web Developer, Back-End Web Developer, Full-Stack Web Developer, Game Developer                                       0.014302
  Mobile Developer,   DevOps / SysAdmin, Data Engineer, Full-Stack Web Developer, Information Security,   User Experience Designer                                               0.014302
  Mobile Developer, Game Developer, Information Security,   Data Scientist                                                                                                       0.014302
Game Developer,   Mobile Developer,   Front-End Web Developer, Back-End Web Developer,   User Experience Designer, Full-Stack Web Developer                                      0.014302
  Quality Assurance Engineer, Information Security                                                                                                                               0.014302
  Mobile Developer,   Data Scientist, Back-End Web Developer, Data Engineer                                                                                                      0.014302
Full-Stack Web Developer, Information Security,   Mobile Developer,   Data Scientist, Game Developer,   Front-End Web Developer                                                  0.014302
Back-End Web Developer, Information Security, Data Engineer, Full-Stack Web Developer,   Data Scientist                                                                          0.014302
Data Engineer, Information Security,   DevOps / SysAdmin, Game Developer                                                                                                         0.014302
Full-Stack Web Developer, Back-End Web Developer,   Mobile Developer, Game Developer,   Front-End Web Developer                                                                  0.014302
Information Security, Game Developer,   Mobile Developer,   DevOps / SysAdmin                                                                                                    0.014302
  Front-End Web Developer, Full-Stack Web Developer,   User Experience Designer,   Mobile Developer, Information Security                                                        0.014302
  Mobile Developer, Data Engineer, Back-End Web Developer, Game Developer,   Front-End Web Developer,   Data Scientist, Full-Stack Web Developer,   User Experience Designer     0.014302
Game Developer,   Front-End Web Developer, Full-Stack Web Developer,   Mobile Developer                                                                                          0.014302
Data Engineer,   Mobile Developer,   Product Manager,   Front-End Web Developer, Game Developer, Back-End Web Developer, Full-Stack Web Developer,   Data Scientist              0.014302
  Mobile Developer, Full-Stack Web Developer,   User Experience Designer, Game Developer                                                                                         0.014302
  Product Manager,   Front-End Web Developer, Full-Stack Web Developer, Information Security                                                                                     0.014302
Back-End Web Developer,   Mobile Developer, Full-Stack Web Developer, Information Security                                                                                       0.014302
Data Engineer,   DevOps / SysAdmin, Back-End Web Developer                                                                                                                       0.014302
Full-Stack Web Developer, Data Engineer, Game Developer,   DevOps / SysAdmin, Information Security,   Data Scientist                                                             0.014302
Data Engineer,   DevOps / SysAdmin,   Data Scientist, Game Developer,   Front-End Web Developer                                                                                  0.014302
  User Experience Designer, Back-End Web Developer,   Mobile Developer, Full-Stack Web Developer,   Front-End Web Developer                                                      0.014302
Information Security,   Product Manager, Full-Stack Web Developer                                                                                                                0.014302
  Quality Assurance Engineer,   Product Manager,   Data Scientist,   Front-End Web Developer                                                                                     0.014302
Full-Stack Web Developer,   Mobile Developer,   DevOps / SysAdmin,   Data Scientist, Back-End Web Developer                                                                      0.014302
Full-Stack Web Developer, Back-End Web Developer,   Data Scientist,   Mobile Developer                                                                                           0.014302
Name: JobRoleInterest, Length: 3213, dtype: float64

The information in the table above is quite granular, but from a quick scan it looks like:

• A lot of people are interested in web development (full-stack web development, front-end web development and back-end web development).
• A few people are interested in mobile development.
• A few people are interested in domains other than web and mobile development.

It’s also interesting to note that many respondents are interested in more than one subject. It’d be useful to get a better picture of how many people are interested in a single subject and how many have mixed interests. Consequently, in the next code block, we’ll:

• Split each string in the JobRoleInterest column to find the number of options for each participant.
  • We’ll first drop the null values because we can’t split Nan values.
  • Generate a frequency table for the variable describing the number of options.

# Split each string in the 'JobRoleInterest' column

interests_no_nulls = survey['JobRoleInterest'].dropna()
split_val = interests_no_nulls.str.split(',')

# Frequency table for the var describing the number of options
n_of_options = split_val.apply(lambda x: len(x))
n_of_options.value_counts(normalize = True).sort_index() * 100

1     31.650458
2     10.883867
3     15.889588
4     15.217391
5     12.042334
6      6.721968
7      3.861556
8      1.759153
9      0.986842
10     0.471968
11     0.185927
12     0.300343
13     0.028604
Name: JobRoleInterest, dtype: float64

It turns out that only 31.7% of the participants have a clear idea about what programming niche they’d like to work in, while the vast majority of students have mixed interests. But given that we offer courses on various subjects, the fact that new coders have mixed interest might be actually good for us.

The focus of our courses is on web and mobile development, so let’s find out how many respondents chose at least one of these two options.

# Frequency table
web_mobile = interests_no_nulls.str.contains(
    'Web Developer|Mobile Developer')

freq_table = web_mobile.value_counts(normalize = True) * 100
print(freq_table)

plt.style.use('fivethirtyeight')
freq_table.plot.bar()
plt.title('Most Participants are Interested in \nWeb or Mobile Development',
          y = 1.08) # y pads the title upward
plt.ylabel('Percentage', fontsize = 12)
plt.xticks([0,1],['Web or mobile\ndevelopment', 'Other subject'],
           rotation = 0) # the initial xtick labels were True and False
plt.ylim([0,100])
plt.show()

True     86.241419
False    13.758581
Name: JobRoleInterest, dtype: float64

It turns out that 86% of the people in our data are interested in web or mobile development. These figures offer us a strong reason to consider this sample representative for our population of interest. We want to advertise our courses to people interested in all sorts of programming niches but mostly web and mobile development.

Now we need to figure out what are the best markets to invest money in for advertising our courses. We’d like to know:

• Where are these new coders located.
• What are the locations with the greatest number of new coders.
• How much money new coders are willing to spend on learning.

Location of new coders

The data set provides information about the location of each participant at a country level. The CountryCitizen variable describes the country of origin for each participant, and the CountryLive variable describes what country each participants lives in (which may be different than the origin country).

We’ll work with the CountryLive variable because we’re interested where people actually live at the moment when we run the ads. Because the data set provides information at a country level, we can think of each country as an individual market. This means we can frame our goal as finding the two best countries to advertise in.

One indicator of a good market is the number of potential customers — the more potential customers in a market, the better. If our ads manage to convince 10% of the 5000 potential customers in market A to buy our product, then this is better than convincing 100% of the 30 potential customers in market B.

survey_no_job_null = survey.dropna(subset=['JobRoleInterest']).copy()

country_abs_freq = survey_no_job_null['CountryLive'].value_counts()
country_rel_freq = survey_no_job_null['CountryLive'].value_counts(normalize = True) * 100

# Display the frequency tables in a more readable format
pd.DataFrame(data = {'Absolute frequency': country_abs_freq, 
                     'Percentage': country_rel_freq}
            )

45.7% of our potential customers are located in the US, and this definitely seems like the most interesting market. India has the second customer density, but it’s just 7.7%, which is not too far from the United Kingdom (4.6%) or Canada (3.8%).

This is useful information, but we need to go more in depth than this and figure out how much money people are actually willing to spend on learning. Advertising in high-density markets where most people are only willing to learn for free is extremely unlikely to be profitable for us.

Spending Money For Learning

The MoneyForLearning column describes in American dollars the amount of money spent by participants from the moment they started coding until the moment they completed the survey. Our company sells subscriptions at a price of $59 per month, and for this reason we’re interested in finding out how much money each student spends per month.

It also seems like a good idea to narrow down our analysis to only four countries: the US, India, the United Kingdom, and Canada. Two reasons for this decision are:

• These are the countries having the highest absolute frequencies in our sample, which means we have a decent amount of data for each.
• Our courses are written in English, and English is an official language in all these four countries. The more people that know English, the better our chances to target the right people with our ads.

survey_no_job_null['MonthsProgramming'].replace(0,1, inplace = True)

survey_no_job_null['MoneyPerMonth'] = survey_no_job_null['MoneyForLearning']/ survey_no_job_null['MonthsProgramming']

print(survey_no_job_null['MoneyPerMonth'].isna().sum())

675

Let’s keep only the rows that don’t have null values for the money_per_month column.

# Keep only the rows with non-nulls in the `money_per_month` column 
month_no_null = survey_no_job_null[survey_no_job_null['MoneyPerMonth'].notnull()]

We want to group the data by country, and then measure the average amount of money that students spend per month in each country. First, let’s remove the rows having null values for the CountryLive column, and check out if we still have enough data for the four countries that interest us.

country_no_null = month_no_null.dropna(subset = ['CountryLive']).copy()
country_no_null['CountryLive'].value_counts()[:4]

United States of America    2933
India                        463
United Kingdom               279
Canada                       240
Name: CountryLive, dtype: int64

This should be enough, so let’s compute the average value spent per month in each country by a student. We’ll compute the average using the mean.

country_mean = country_no_null[['CountryLive','MoneyPerMonth']].groupby('CountryLive').mean()
country_mean['MoneyPerMonth'][['United States of America',
                            'India', 'United Kingdom',
                            'Canada']]

CountryLive
United States of America    227.997996
India                       135.100982
United Kingdom               45.534443
Canada                      113.510961
Name: MoneyPerMonth, dtype: float64

The results for the United Kingdom and Canada are a bit surprising relative to the values we see for India. If we considered a few socio-economical metrics (like GDP per capita), we’d intuitively expect people in the UK and Canada to spend more on learning than people in India.

It might be that we don’t have have enough representative data for the United Kingdom and Canada, or we have some outliers (maybe coming from wrong survey answers) making the mean too large for India, or too low for the UK and Canada. Or it might be that the results are correct.

Dealing with Outliers

Let’s use box plots to visualize the distribution of the MoneyPerMonth variable for each country.

#Isolate the top four countries

only_4 = country_no_null[country_no_null['CountryLive'].str.contains(
    'United States of America|India|United Kingdom|Canada')]

import seaborn as sns

sns.boxplot(data = only_4, x = 'CountryLive', y = 'MoneyPerMonth')
plt.title('Money Spent Per Month Per Country\n(Distributions)',
         fontsize = 16)
plt.ylabel('Money per month (US dollars)')
plt.xlabel('Country')
plt.xticks(range(4), ['US', 'UK', 'India', 'Canada']) # avoids tick labels overlap
plt.show()

It’s hard to see on the plot above if there’s anything wrong with the data for the United Kingdom, India, or Canada, but we can see immediately that there’s something really off for the US: two persons spend each month \$50000 or more for learning. This is not impossible, but it seems extremely unlikely, so we’ll remove every value that goes over \$20,000 per month.

# Isolate only those participants who spend less than 20000 per month
country_no_null = country_no_null[country_no_null['MoneyPerMonth'] < 20000]

country_newmean = country_no_null[['CountryLive','MoneyPerMonth']].groupby('CountryLive').mean()
country_newmean['MoneyPerMonth'][['United States of America',
                            'India', 'United Kingdom',
                            'Canada']]

CountryLive
United States of America    183.800110
India                       135.100982
United Kingdom               45.534443
Canada                      113.510961
Name: MoneyPerMonth, dtype: float64

new_only_4 = country_no_null[country_no_null['CountryLive'].str.contains(
    'United States of America|India|United Kingdom|Canada')]

sns.boxplot(data = new_only_4, x = 'CountryLive', y = 'MoneyPerMonth')
plt.title('Money Spent Per Month Per Country\n(Distributions)',
         fontsize = 16)
plt.ylabel('Money per month (US dollars)')
plt.xlabel('Country')
plt.xticks(range(4), ['US', 'UK', 'India', 'Canada']) # avoids tick labels overlap
plt.show()

We can see a few extreme outliers for India (values over $2500 per month), but it’s unclear whether this is good data or not. Maybe these people attended several bootcamps, which tend to be very expensive. Let’s examine these two data points to see if we can find anything relevant.

# Inspect the extreme outliers for India
india_outliers = new_only_4[
    (only_4['CountryLive'] == 'India') & 
    (only_4['MoneyPerMonth'] >= 2500)]
india_outliers

It seems that neither participant attended a bootcamp. Overall, it’s really hard to figure out from the data whether these people really spent that much money with learning. The actual question of the survey was “Aside from university tuition, about how much money have you spent on learning to code so far (in US dollars)?”, so they might have misunderstood and thought university tuition is included. It seems safer to remove these four rows.

# Remove the outliers for India
only_4 = new_only_4.drop(india_outliers.index) # using the row labels

Looking back at the box plot above, we can also see more extreme outliers for the US (values over $6000 per month). Let’s examine these participants in more detail.

# Examine the extreme outliers for the US
us_outliers = only_4[
    (only_4['CountryLive'] == 'United States of America') & 
    (only_4['MoneyPerMonth'] >= 6000)]

us_outliers

Out of these 11 extreme outliers, six people attended bootcamps, which justify the large sums of money spent on learning. For the other five, it’s hard to figure out from the data where they could have spent that much money on learning. Consequently, we’ll remove those rows where participants reported thay they spend \$6000 each month, but they have never attended a bootcamp.

Also, the data shows that eight respondents had been programming for no more than three months when they completed the survey. They most likely paid a large sum of money for a bootcamp that was going to last for several months, so the amount of money spent per month is unrealistic and should be significantly lower (because they probably didn’t spend anything for the next couple of months after the survey). As a consequence, we’ll remove every these eight outliers.

In the next code block, we’ll remove respondents that:

• Didn’t attend bootcamps.
• Had been programming for three months or less when at the time they completed the survey.

# Remove the respondents who didn't attendent a bootcamp
no_bootcamp = only_4[
    (only_4['CountryLive'] == 'United States of America') & 
    (only_4['MoneyPerMonth'] >= 6000) &
    (only_4['AttendedBootcamp'] == 0)
]

only_4 = only_4.drop(no_bootcamp.index)

# Remove the respondents that had been programming for less than 3 months
less_than_3_months = only_4[
    (only_4['CountryLive'] == 'United States of America') & 
    (only_4['MoneyPerMonth'] >= 6000) &
    (only_4['MonthsProgramming'] <= 3)
]

only_4 = only_4.drop(less_than_3_months.index)

Looking again at the last box plot above, we can also see an extreme outlier for Canada — a person who spends roughly $5000 per month. Let’s examine this person in more depth.

# Examine the extreme outliers for Canada
canada_outliers = only_4[
    (only_4['CountryLive'] == 'Canada') & 
    (only_4['MoneyPerMonth'] > 4500)]

canada_outliers

Here, the situation is similar to some of the US respondents — this participant had been programming for no more than two months when he completed the survey. He seems to have paid a large sum of money in the beginning to enroll in a bootcamp, and then he probably didn’t spend anything for the next couple of months after the survey. We’ll take the same approach here as for the US and remove this outlier.

# Remove the extreme outliers for Canada
only_4 = only_4.drop(canada_outliers.index)

# Recompute mean sum of money spent by students each month
only_4.groupby('CountryLive').mean()['MoneyPerMonth']

CountryLive
Canada                       93.065400
India                        65.758763
United Kingdom               45.534443
United States of America    142.654608
Name: MoneyPerMonth, dtype: float64

# Visualize the distributions again
sns.boxplot(y = 'MoneyPerMonth', x = 'CountryLive',
            data = only_4)
plt.title('Money Spent Per Month Per Country\n(Distributions)',
          fontsize = 16)
plt.ylabel('Money per month (US dollars)')
plt.xlabel('Country')
plt.xticks(range(4), ['US', 'UK', 'India', 'Canada']) # avoids tick labels overlap
plt.show()

Choosing the Two best Markets

Obviously, one country we should advertise in is the US. Lots of new coders live there and they are willing to pay a good amount of money each month (roughly \$143).

We sell subscriptions at a price of \$59 per month, and Canada seems to be the best second choice because people there are willing to pay roughly \$93 per month, compared to India (\$66) and the United Kingdom (\$45).

The data suggests strongly that we shouldn’t advertise in the UK, but let’s take a second look at India before deciding to choose Canada as our second best choice:

• \$59 doesn’t seem like an expensive sum for people in India since they spend on average \$66 each month.
• We have almost twice as more potential customers in India than we have in Canada:

# Frequency table for the 'CountryLive' column
only_4['CountryLive'].value_counts()

United States of America    2920
India                        457
United Kingdom               279
Canada                       239
Name: CountryLive, dtype: int64

So it’s not crystal clear what to choose between Canada and India. Although it seems more tempting to choose Canada, there are good chances that India might actually be a better choice because of the large number of potential customers.

At this point, it seems that we have several options:

• Advertise in the US, India, and Canada by splitting the advertisement budget in various combinations:
• 60% for the US, 25% for India, 15% for Canada.
• 50% for the US, 30% for India, 20% for Canada; etc.

Advertise only in the US and India, or the US and Canada. Again, it makes sense to split the advertisement budget unequally. For instance: * 70% for the US, and 30% for India. * 65% for the US, and 35% for Canada; etc. * Advertise only in the US.

At this point, it’s probably best to send our analysis to the marketing team and let them use their domain knowledge to decide. They might want to do some extra surveys in India and Canada and then get back to us for analyzing the new survey data.

Conclusion

In this project, we analyzed survey data from new coders to find the best two markets to advertise in. The only solid conclusion we reached is that the US would be a good market to advertise in.

For the second best market, it wasn’t clear-cut what to choose between India and Canada. We decided to send the results to the marketing team so they can use their domain knowledge to take the best decision.