Posts

Missing Data : What to Do?

Image
  A missing tooth: that's not good (or is it?). Today's topic is inspired by a problem that one of my coworkers brought to my attention, and I think this is also something many people face on a day-to-day basis. As data professionals, we have all encountered datasets that contain fields with no data, which can be frustrating when building reports or machine learning models. Since we all know that a perfect dataset does not exist in the real world, the next best thing to do would be to have a game plan in place to tackle the inevitable. When posed with the question, "Toni, how should I go about solving this?" the first word that comes to mind is context. You see, just because something is missing doesn't mean it needs to be "fixed" for example, knowing why a tooth a missing is important before acting.  A 6-year old with a missing tooth doesn't require a filling.  With that being said, here is a strategy for dealing with the unknown: Why is the data no...

Metadata : Get To Know Your Dataset

Image
  Price tags provide metadata about the product they are attached to. The "meta-" part of metadata means "beyond" or "transcending." So, metadata is like extra information that goes beyond the main data. For example, think about your name, personality type, height, and weight.  These are all things that help people know more about you. Here's an example some of my metadata: Name: Toni-Marie Noel Personality Type: ENTJ Height: 5 feet 6 inches (167.64 cm) Weight: 129 pounds (58.51 kg) Favorite Pastry: Currants Roll  I first heard about metadata at work, and it's becoming a really important aspect of my role. But why is metadata important in the first place? Searchability Imagine searching for George Orwell's 'Animal Farm' in a library filled with books lacking labels on their covers. A nightmare, I know! That's why I actually judge books by their cover. Search Engine Optimization  Utilizing the correct keywords can lead to a stagge...

SQL Server Reporting Services vs. Power BI

Image
  When I think of Reporting, I think of the news.  Throughout my career in Technology, I have encountered two Microsoft products, Power BI and SQL Server Reporting Services (SSRS), both designed to serve the same purpose: Reporting.  Today, I decided to delve into their origins and explore their respective strengths and weaknesses. A Brief History on SSRS  If you were to ask me, "What is SSRS?" before my current job, I'd probably just stare blankly and shrug my shoulders. However, now I can confidently tell you that SSRS stands for SQL Server Reporting Services. Here's a fun fact: It started in 2004 as an add-on for SQL Server 2000. People really liked it because it could work with all sorts of data, such as tables from SQL Server databases, Excel, and XML files. And you know what's interesting? Even after all these years, people are still using it. SSRS Strengths SSRS is better suited for generating paginated reports, such as invoices and statements. Report subs...

Internal Customers: How to Better Serve Them?

Image
  "The best way to find yourself is to lose yourself in the service of others." - Mahatma Gandhi I began my career by engaging with external customers over the phone. It always started with introducing myself, stating my location, and the purpose of the call, all of which were recorded for quality assurance.  It would often go something like :  "Hi, my name is Toni-Marie and I am calling from Mars, I would like to speak to Mr. Salad Fingers"   I said that phrase so often that I would automatically say it, even when placing an order at Mario's: "Hi, my name is Toni-Marie and I would like to order a Chicken Picante Sandwich"  As part of the Technology Department, I now serve internal customers, and although these individuals work at the same place, I still believe that there needs to be a sense of urgency and respect when dealing with them.  Their perception of the Technology Department can impact the overall performance of the company.  If they do not t...

Choosing the Right Project for Your Portfolio

Image
   Etymologically, the word "project" essentially means "to throw or cast forward."  As students, we have all been assigned multiple projects to complete within in a stipulated time period whether we liked them or not, from papier-mâché in art class to solar system models in science.  We completed these projects for good grades at the end of the term, which may have led to an even greater incentive, that puppy you wanted for your birthday.  Moving into our professional lives as technology experts, we often face new projects, but now there's a paycheck waiting for us at the end.  This is great because we all have bills to pay.  But, if we focus only on the money, we might not be so willing to put in extra effort without getting paid for it. Recently, I began watching One Piece, and a common thread among all the main characters that stands out is their dedication to something they genuinely love.  Take Sanji, for example, who is so passionate about ...

Web Scraping: An Introduction

Image
  Wolverine was here Web scraping has been used to compile extensive datasets on everything from UFO sightings to product reviews on the internet! If you have ever gone to a website, copied data from it, and pasted it into an Excel file, then congratulations, you have done web scraping manually. However, what if you had a bigger problem to solve and required more data from a website? For example, the wristband for your cheap Casio watch broke, and you decided it's time to upgrade to a more premium model.  However, before spending your money, you wanted to compare the prices, reviews and materials of thousands of watches to ensure that you are making the best decision when it comes to durability and your budget.  I hope we can both agree that this task is humanly impossible to get done within a reasonable amount of time, and that's where libraries like Scrapy and Beautiful Soup come into play. With great power comes great responsibility! Here are some pros and cons of web ...

Validating Email Address Format with Python's Great Expectations

Image
  Your email address is valid! This week, I had the task of gathering a client contact list, but I ran into some issues with certain email addresses. Specifically, I noticed that some email addresses were missing the '@' symbol, while others were lacking the domain name. To ensure the integrity of our client contact list, it's crucial that all email addresses follow the expected format: username@domain.com. This way, when we use this data for its intended purpose – sending emails to customers regarding promotions, updates, or other important communications – we can minimize the number of undeliverable emails. I've already taken steps to address this issue by removing rows that don't adhere to this format using Alteryx.  Today, I'm excited to show you how we can use Great Expectations on Google Colaboratory to identify  erroneous emails. I want to give a special shout-out to our Product Manager, Ikechi Griffith, for challenging me to dive deeper into Great Expect...