Data Slug

•

Vintage floppy disk on wooden surface — Data Slug blog header image

Journey over Data Tools and Exploration

https://www.python.org/

Python was not my first programming language but I’m sure it won’t be my last either. Yet it was more than a decade ago I got my first glimpse of what it can do. Coming from a C based curriculum and mental model, Python gave me the tools I need to develop things that people can use everyday and not use archaic forms of electronic scripts.

I have this hope that there is a better way. Higher-level tools that actually let you see the structure of the software more clearly will be of tremendous value.

– Guido van Rossum, Inventor of Python

Fast forward a bit, these days I found myself needing to build internal tools within the company as well as quickly trying to iterate ideas and Python has become the go-to tools for everything that I do. While the fundamentals of the programming hasn’t changed, it allows myself to iterate ideas faster rather than being stuck on the crud of environment setups where it has been the many of early developers on any language.

Streamlit

https://streamlit.io/

In the recent years, I have found that in small company and startups, you are drowned in piles of terabytes of unorganized data with minimal data lineage or maintenance, you resorted to bash scripts, magic one-liners Perl scripts copied from Stack Overflow, and one-offs uncommitted Python scripts. This was before LLM become something of a common thing, so one case that I keep tripping in terms of maintenance and initial bootstrap was. a web tool.

While there are many good framework for developing frontend or backend in Python, I found myself to have a slow start in putting them together. My responsibilities do not allow me dedicated coding time, but seeing dire need I started to explore possible tool to iterating fast for dashboard with accessible interfaces, which is a web browser.

I got the buy-in when I found Streamlit early version on promises on complete Pythonic web applications. I found using combination of Pandas and Numpy allows me to do what I can normally do with Jupyter notebook, but quickly make it accessible tool to others without being too pedantic on my Javascript boot strap.

import pandas as pd
import streamlit as st
d = pd.read_csv('dataset.csv')
st.table(d)

Streamlit allows for the complexity of doing a render on inspecting data or just simply showing it on a web frontend. By running streamlit run, you can render the dataset and show it on a tabular form (like above) or on a plot. Building on that, you start creating web tools that are easily accessible rather than just a purely research tool.

streamlit run <app.py>

Whilst there are other tools to do this before like Plotly Dash, Streamlit is born out of new framework of rapid tools development. Even further back, it would requires separate development of front end and back end applications to be tied that create complex development cycles for sharing quick tools internally especially in small teams environments.

I hope this introduction to Python and Streamlit, inspires you to explore usage of Python and Streamlit for data exploration, especially if you are an early data analyst or business analyst. Thank you for reading until the end.

4 responses to “Python and Streamlit”

  1. […] for this purpose, another great tool to quickly prototype your web dashboard. I have done earlier post where I show the basic power developing dashboard. On top of that, the framework is established […]

  2. Daisuke Yoshida Avatar
    Daisuke Yoshida

    I didn’t know about Streamlit. It’s pretty cool. For RAG I have learned that the AI is doesn’t retrieve the right chunk sometimes, which could be due to chunk size and chunk overlap, could be relevent chunk is not retrieved by AI itself, or could be due to the AI model. We may still need to double check with our own eyes 😀

    1. I’ve dig deeper a bit on this. Semantic meaning can be very domain specialized context – like legal, finance, HR, and company specific jargons. Embedding models out there are usually meant for general use case, so additional techniques are needed depending on what you are trying to do.

  3. […] for this purpose, another great tool to quickly prototype your web dashboard. I have done earlier post where I show the basic power developing dashboard. On top of that, the framework is established […]

Leave a Reply