Most Flutter developers build AI chat interfaces like regular chat apps. They collect the full response, then display it all at once. But AI responses aren’t like human messages—they stream in token by token, creating that characteristic “typing” effect that users expect from ChatGPT, Claude, and other AI assistants.
The problem isn’t just user experience. When you wait for complete responses before updating your UI, users stare at loading spinners for 10-20 seconds. They assume your app is frozen and start tapping frantically. Meanwhile, your AI provider is already streaming the first words of the answer.
I’ve seen Flutter developers try to solve this with setState, updating the UI every time a new token arrives. The result? Janky animations, dropped frames, and chat bubbles that grow and shrink unpredictably. There’s a better way.
The setState Trap
Here’s how most developers first attempt streaming AI responses:
class ChatScreen extends StatefulWidget {
@override
_ChatScreenState createState() => _ChatScreenState();
}
class _ChatScreenState extends State<ChatScreen> {
String currentResponse = '';
void sendMessage() async {
final stream = openAI.sendMessageStream('Hello, AI!');
await for (final token in stream) {
setState(() {
currentResponse += token; // This triggers rebuild on every token
});
}
}
@override
Widget build(BuildContext context) {
return Column(
children: [
// Chat history
ChatBubble(text: currentResponse), // Rebuilds constantly
],
);
}
}
This approach has three major problems:
- Performance: setState rebuilds the entire widget tree on every token
- Visual glitches: Chat bubbles resize constantly, causing layout shifts
- Lost scroll position: ListView jumps around as content changes
Every token arrival triggers a full rebuild. If your AI streams 100 tokens, that’s 100 widget rebuilds for a single response.
StreamBuilder: The Right Tool for Streaming Data
Flutter gives us StreamBuilder specifically for handling streaming data. Instead of managing state manually, let StreamBuilder subscribe to your AI response stream and rebuild only the relevant parts of your UI.
class StreamingChatBubble extends StatelessWidget {
final Stream<String> responseStream;
const StreamingChatBubble({required this.responseStream});
@override
Widget build(BuildContext context) {
return StreamBuilder<String>(
stream: responseStream,
initialData: '',
builder: (context, snapshot) {
final text = snapshot.data ?? '';
return Container(
padding: EdgeInsets.all(12),
margin: EdgeInsets.symmetric(vertical: 4, horizontal: 8),
decoration: BoxDecoration(
color: Colors.grey[100],
borderRadius: BorderRadius.circular(12),
),
child: Column(
crossAxisAlignment: CrossAxisAlignment.start,
children: [
Text(
text,
style: TextStyle(fontSize: 16),
),
if (snapshot.connectionState == ConnectionState.waiting)
Container(
margin: EdgeInsets.only(top: 8),
child: Row(
children: [
SizedBox(
width: 12,
height: 12,
child: CircularProgressIndicator(strokeWidth: 2),
),
SizedBox(width: 8),
Text(
'AI is typing...',
style: TextStyle(
fontSize: 12,
color: Colors.grey[600],
),
),
],
),
),
],
),
);
},
);
}
}
This StreamBuilder approach gives you:
- Isolated rebuilds: Only the chat bubble rebuilds, not the entire screen
- Connection state awareness: Show typing indicators while streaming
- Error handling: Built-in error states for network issues
- Clean completion: Automatically stops listening when stream closes
Building a Complete Streaming Chat Interface
Let’s build a complete chat interface that handles multiple streaming responses elegantly. We’ll use a combination of StreamBuilder for active responses and a regular list for completed messages.
class StreamingChatInterface extends StatefulWidget {
@override
_StreamingChatInterfaceState createState() => _StreamingChatInterfaceState();
}
class _StreamingChatInterfaceState extends State<StreamingChatInterface> {
final TextEditingController _controller = TextEditingController();
final List<ChatMessage> _messages = [];
Stream<String>? _currentResponseStream;
void _sendMessage() async {
final userMessage = _controller.text.trim();
if (userMessage.isEmpty) return;
setState(() {
_messages.add(ChatMessage(
text: userMessage,
isUser: true,
isComplete: true,
));
_controller.clear();
});
// Start streaming AI response
final responseStream = _getAIResponseStream(userMessage);
setState(() {
_currentResponseStream = responseStream;
});
// Collect the complete response
String completeResponse = '';
await for (final token in responseStream) {
completeResponse += token;
}
// Replace streaming response with final message
setState(() {
_currentResponseStream = null;
_messages.add(ChatMessage(
text: completeResponse,
isUser: false,
isComplete: true,
));
});
}
Stream<String> _getAIResponseStream(String prompt) async* {
// Simulate AI response streaming
final words = 'This is a simulated AI response that streams word by word to demonstrate the streaming chat interface.'.split(' ');
for (int i = 0; i < words.length; i++) {
await Future.delayed(Duration(milliseconds: 100));
if (i == 0) {
yield words[i];
} else {
yield ' ${words[i]}';
}
}
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(title: Text('Streaming AI Chat')),
body: Column(
children: [
Expanded(
child: ListView.builder(
reverse: true,
itemCount: _messages.length + (_currentResponseStream != null ? 1 : 0),
itemBuilder: (context, index) {
// Show streaming response at the top
if (index == 0 && _currentResponseStream != null) {
return StreamingChatBubble(
responseStream: _currentResponseStream!,
isUser: false,
);
}
// Show completed messages
final messageIndex = _currentResponseStream != null ? index - 1 : index;
final message = _messages[_messages.length - 1 - messageIndex];
return ChatBubble(
text: message.text,
isUser: message.isUser,
);
},
),
),
Container(
padding: EdgeInsets.all(16),
child: Row(
children: [
Expanded(
child: TextField(
controller: _controller,
decoration: InputDecoration(
hintText: 'Type a message...',
border: OutlineInputBorder(
borderRadius: BorderRadius.circular(20),
),
),
onSubmitted: (_) => _sendMessage(),
),
),
SizedBox(width: 8),
FloatingActionButton(
onPressed: _sendMessage,
mini: true,
child: Icon(Icons.send),
),
],
),
),
],
),
);
}
}
class ChatMessage {
final String text;
final bool isUser;
final bool isComplete;
ChatMessage({
required this.text,
required this.isUser,
required this.isComplete,
});
}
class StreamingChatBubble extends StatelessWidget {
final Stream<String> responseStream;
final bool isUser;
const StreamingChatBubble({
required this.responseStream,
required this.isUser,
});
@override
Widget build(BuildContext context) {
return StreamBuilder<String>(
stream: responseStream,
initialData: '',
builder: (context, snapshot) {
return ChatBubble(
text: snapshot.data ?? '',
isUser: isUser,
isStreaming: snapshot.connectionState == ConnectionState.active,
);
},
);
}
}
class ChatBubble extends StatelessWidget {
final String text;
final bool isUser;
final bool isStreaming;
const ChatBubble({
required this.text,
required this.isUser,
this.isStreaming = false,
});
@override
Widget build(BuildContext context) {
return Container(
margin: EdgeInsets.symmetric(vertical: 4, horizontal: 16),
child: Row(
mainAxisAlignment: isUser ? MainAxisAlignment.end : MainAxisAlignment.start,
children: [
Flexible(
child: Container(
padding: EdgeInsets.symmetric(horizontal: 16, vertical: 12),
decoration: BoxDecoration(
color: isUser ? Colors.blue[500] : Colors.grey[200],
borderRadius: BorderRadius.circular(16),
),
child: Column(
crossAxisAlignment: CrossAxisAlignment.start,
children: [
Text(
text,
style: TextStyle(
color: isUser ? Colors.white : Colors.black87,
fontSize: 16,
),
),
if (isStreaming)
Container(
margin: EdgeInsets.only(top: 8),
child: SizedBox(
width: 12,
height: 12,
child: CircularProgressIndicator(
strokeWidth: 2,
valueColor: AlwaysStoppedAnimation<Color>(
isUser ? Colors.white : Colors.grey[600]!,
),
),
),
),
],
),
),
),
],
),
);
}
}
Performance Considerations and Best Practices
When implementing streaming AI responses, keep these performance tips in mind:
1. Debounce Token Updates
If your AI provider sends tokens very rapidly, consider debouncing updates to avoid excessive rebuilds:
Timer? _debounceTimer;
String _accumulatedTokens = '';
void _handleToken(String token) {
_accumulatedTokens += token;
_debounceTimer?.cancel();
_debounceTimer = Timer(Duration(milliseconds: 50), () {
// Update UI with accumulated tokens
setState(() {
currentResponse = _accumulatedTokens;
});
});
}
2. Memory Management
Long conversations can accumulate large amounts of data. Consider implementing message pagination or limiting the number of visible messages:
class ChatController {
static const int maxVisibleMessages = 100;
final List<ChatMessage> _allMessages = [];
List<ChatMessage> get visibleMessages {
if (_allMessages.length <= maxVisibleMessages) {
return _allMessages;
}
return _allMessages.sublist(_allMessages.length - maxVisibleMessages);
}
}
3. Error Handling
Streams can fail. Always handle errors gracefully:
StreamBuilder<String>(
stream: responseStream,
builder: (context, snapshot) {
if (snapshot.hasError) {
return ChatBubble(
text: 'Sorry, something went wrong. Please try again.',
isUser: false,
isError: true,
);
}
return ChatBubble(
text: snapshot.data ?? '',
isUser: false,
isStreaming: snapshot.connectionState == ConnectionState.active,
);
},
)
Connecting to Real AI Providers
The examples above use simulated streaming, but connecting to real AI providers follows the same pattern. Here’s how to integrate with OpenAI’s streaming API:
Stream<String> _getOpenAIStream(String prompt) async* {
final client = http.Client();
try {
final request = http.Request('POST', Uri.parse('https://api.openai.com/v1/chat/completions'));
request.headers.addAll({
'Authorization': 'Bearer $openAIKey',
'Content-Type': 'application/json',
});
request.body = json.encode({
'model': 'gpt-3.5-turbo',
'messages': [{'role': 'user', 'content': prompt}],
'stream': true,
});
final streamedResponse = await client.send(request);
await for (final chunk in streamedResponse.stream.transform(utf8.decoder)) {
final lines = chunk.split('n');
for (final line in lines) {
if (line.startsWith('data: ') && !line.contains('[DONE]')) {
final data = line.substring(6);
try {
final json = jsonDecode(data);
final content = json['choices'][0]['delta']['content'];
if (content != null) {
yield content;
}
} catch (e) {
// Skip malformed chunks
}
}
}
}
} finally {
client.close();
}
}
Next Steps
You now have a solid foundation for building streaming AI chat interfaces in Flutter. The key insight is using StreamBuilder instead of setState—it gives you better performance, cleaner code, and built-in connection state management.
From here, consider adding:
- Message persistence: Save conversations to local storage
- Rich message types: Support images, code blocks, and markdown
- Conversation branching: Let users retry or edit previous messages
- Voice input: Add speech-to-text for voice conversations
The patterns shown here scale to complex chat applications. Whether you’re building a customer service bot or an AI assistant, StreamBuilder gives you the foundation for smooth, responsive streaming interfaces.
Remember: AI responses should feel immediate and natural. Users expect that characteristic streaming effect, and with the right implementation, your Flutter app can deliver it without performance compromises.