Adding streaming and a persona
Two upgrades that take the app from "okay" to "feels like ChatGPT". Both are 5 minutes of work.
Adding wheels to a chair. Tiny change, totally different experience.
Streaming: convert the OpenAI call to stream=True and update the UI as tokens arrive.
Persona: prepend a system message to st.session_state.messages.
Replace the API call section with:
SYSTEM = (
"You are GeekBot, a friendly tech tutor for beginners. "
"Reply in short, clear paragraphs. Use simple analogies."
)
# Ensure system message is first
if not st.session_state.messages or st.session_state.messages[0]["role"] != "system":
st.session_state.messages.insert(0, {"role": "system", "content": SYSTEM})
if prompt := st.chat_input("Ask me anything"):
st.session_state.messages.append({"role": "user", "content": prompt})
with st.chat_message("user"):
st.markdown(prompt)
with st.chat_message("assistant"):
placeholder = st.empty()
stream = client.chat.completions.create(
model="gpt-4o-mini",
messages=st.session_state.messages,
stream=True,
)
text = ""
for chunk in stream:
delta = chunk.choices[0].delta.content
if delta:
text += delta
placeholder.markdown(text + "_")
placeholder.markdown(text)
st.session_state.messages.append({"role": "assistant", "content": text})
Done. Live streaming and a custom persona.
Quick recall
3 prompts · think before you flip
Prompt 1 of 3
How does the placeholder pattern work?
Quiz time
1 question · tap an answer to check it
1. The "_" suffix on `placeholder.markdown(text + "_")` is for
Finished lesson 6.3?
Mark complete to update your module progress and unlock the streak.
Loading