We usually print the outputs to check if our logic runs as expected when it comes to coding. However, runtime issues may arise during production where we can’t verify the print outputs. That’s when logging helps us to identify the runtime issues. Let’s get started!
Create a python file
log_utility.py and write the first line of code:
Simple, eh? Logging is one of the essential features of software development, so it’s nice of Python to have this built-in module to support serious software development.
log_utility.py, we define a function and initialize all the parameters:
def get_logger(name, log_config…
As a data scientist, I’m always interested to know how certain independent variables, such as occupation, influence the dependent variables, such as income. Specifically, when it comes to classification problems, WoE and IV can tell stories between an independent variable and a dependent variable.
These concepts were developed mainly to answer credit scoring problems, where customers are labelled either ‘good’ or ‘bad’, which is based on them defaulting credit repayment. Their associated variables, such as age, are also recorded.
Now, we are applying these concepts on the Titanic data set.
using DataFrames, CSV
df = DataFrame(CSV.File("train.csv"))
In order to install npm, install Node.js and npm will come with it. Then, start ‘Command Prompt (cmd)’, point to the…
This continues from Part 1 where I walk through the steps of Business Understanding and Data Understanding. The actions taken are formulating problem statements, web scraping for data and storing the data in a database.
For Part 2, we focus on Data Preparation and Data Modelling!
Obviously, we first fetch the data from the database that stores them and randomly sample 5 observations of our career data.
engine = create_engine('postgresql://[username]:[password]@[database].ap-southeast-1.rds.amazonaws.com:5432/postgres', echo=True)
query = """
SELECT * FROM posting;
posting = pd.read_sql(query, engine)
“Let’s jump into modelling!” I know how eager we can be when it comes to modelling…
Endless challenges. That’s how we grow. In our Data Science Immersive program, the last major project before the Capstone is to build predictive models for various aspects of job hunting, such as salary and job categories. The project resembles the real-world scenario: Your boss gives you a target and/or a problem statement and you find a way to get it done.