GeekHub Learn
Module
Lesson 6.33 of 7 in this module2 min read Module 6: Building Your First AI Chat App

Adding streaming and a persona

Two upgrades that take the app from "okay" to "feels like ChatGPT". Both are 5 minutes of work.

Adding wheels to a chair. Tiny change, totally different experience.

Streaming: convert the OpenAI call to stream=True and update the UI as tokens arrive.

Persona: prepend a system message to st.session_state.messages.

Replace the API call section with:

SYSTEM = (
    "You are GeekBot, a friendly tech tutor for beginners. "
    "Reply in short, clear paragraphs. Use simple analogies."
)

# Ensure system message is first
if not st.session_state.messages or st.session_state.messages[0]["role"] != "system":
    st.session_state.messages.insert(0, {"role": "system", "content": SYSTEM})

if prompt := st.chat_input("Ask me anything"):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)

    with st.chat_message("assistant"):
        placeholder = st.empty()
        stream = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=st.session_state.messages,
            stream=True,
        )
        text = ""
        for chunk in stream:
            delta = chunk.choices[0].delta.content
            if delta:
                text += delta
                placeholder.markdown(text + "_")
        placeholder.markdown(text)
        st.session_state.messages.append({"role": "assistant", "content": text})

Done. Live streaming and a custom persona.

Visualize it

A GIF idea: side-by-side recording, before and after streaming. Before: blank for 3 seconds then full text. After: text flows in.

Try it now

Edit the system prompt to make GeekBot rude. Re-run. Note how the entire feel of the app changes from one variable.

Hands-on lab

Add streaming and persona to your app. Push to GitHub.

Try it now

Why is the system message inserted at index 0 and not appended?

Common mistakes

  • Forgetting to update placeholder.markdown inside the loop (UI does not stream)
  • Adding multiple system messages over time (clutters context)
  • Putting the persona in the user message instead of system

Debugging tip

If streaming flickers, you may be re-rendering the chat history too aggressively. Streamlit handles this well in chat_message blocks but watch for nested expensive components.

Challenge

Add a sidebar dropdown that lets the user pick from 4 personas. Switch live.

Where this shows up

  • Personality-driven chatbots
  • Branded support assistants
  • Multi-tone copywriting tools

From the field

The two-feature combo of streaming + strong persona is the single biggest "feels like a real product" upgrade in any LLM app.

Recap

Streaming for speed, system prompt for soul. Five minutes. Twice the app.


Quick recall

3 prompts · think before you flip

Prompt 1 of 3

How does the placeholder pattern work?

Quiz time

1 question · tap an answer to check it

  1. 1. The "_" suffix on `placeholder.markdown(text + "_")` is for

Finished lesson 6.3?

Mark complete to update your module progress and unlock the streak.

Loading