useInfiniteScroll Hook: Building Scalable Paginated Lists
Building truly scalable infinite scroll is deceptively complex. Most implementations fail under real-world conditions: duplicate items appear, loading triggers incorrectly, errors go unhandled, networks are slow, and users scroll faster than data loads. The difference between a working infinite scroll and one that feels responsive comes from handling edge cases that most developers discover through painful debugging.
The useInfiniteScroll hook abstracts this complexity, managing pagination state, detecting when to load more, handling errors and retries, and preventing race conditions. Whether you're building a social feed at ByteDance's scale or a product listing at Alibaba's, this hook handles the invisible work that makes scroll feel natural.
Table of Contents
- The Infinite Scroll Challenge
- Basic useInfiniteScroll Implementation
- Advanced Patterns: Retries and Error Recovery
- Intersection Observer Integration
- Cursor-Based vs Offset-Based Pagination
- Practical Application Scenarios
- Performance and Scaling
- FAQ
The Infinite Scroll Challenge
Most first attempts at infinite scroll look like this:
// ❌ This has several bugs
function Feed() {
const [items, setItems] = useState([]);
const [loading, setLoading] = useState(false);
useEffect(() => {
const handleScroll = () => {
if (window.scrollY + window.innerHeight > document.scrollingElement.scrollHeight - 500) {
setLoading(true);
}
};
window.addEventListener('scroll', handleScroll);
return () => window.removeEventListener('scroll', handleScroll);
}, []);
useEffect(() => {
if (!loading) return;
// Problem 1: Race conditions - what if user scrolls before this finishes?
fetch(`/api/items?page=${items.length / 20}`)
.then(res => res.json())
.then(data => {
// Problem 2: Duplicates if scroll fires multiple times
setItems([...items, ...data.items]);
setLoading(false);
})
.catch(err => {
// Problem 3: No error handling
console.error(err);
});
}, [loading]);
return (
<div>
{items.map(item => (
<div key={item.id}>{item.title}</div>
))}
{loading && <div>Loading...</div>}
</div>
);
}
This code suffers from:
- Race conditions: Multiple scroll events queue simultaneous requests
- Duplicate items: Loading flag doesn't prevent double-fetching
- Missing error handling: Network failures silently fail or break state
- No retry logic: User can't recover from transient errors
- Memory issues: No cleanup, no limit on cached items
- Accessibility: No keyboard support, no focus management
A production-grade hook prevents all of these.
Basic useInfiniteScroll Implementation
TypeScript Version
import { useCallback, useEffect, useRef, useState } from 'react';
interface InfiniteScrollState<T> {
items: T[];
isLoading: boolean;
error: Error | null;
hasMore: boolean;
cursor?: string | number;
}
interface UseInfiniteScrollOptions<T> {
// Function to fetch items. Should return { items, nextCursor?, hasMore? }
fetchItems: (cursor?: string | number) => Promise<{
items: T[];
nextCursor?: string | number;
hasMore?: boolean;
}>;
// Initial cursor value
initialCursor?: string | number;
// Fire fetch when user scrolls within this many pixels of bottom
threshold?: number;
// Max items to keep in memory (for performance)
maxItems?: number;
}
export function useInfiniteScroll<T extends { id: string | number }>(
options: UseInfiniteScrollOptions<T>
): InfiniteScrollState<T> & {
retry: () => void;
reset: () => void;
} {
const {
fetchItems,
initialCursor,
threshold = 500,
maxItems = 1000,
} = options;
const [state, setState] = useState<InfiniteScrollState<T>>({
items: [],
isLoading: false,
error: null,
hasMore: true,
cursor: initialCursor,
});
// Track pending request to prevent race conditions
const pendingRequestRef = useRef<Promise<any> | null>(null);
const abortControllerRef = useRef<AbortController | null>(null);
// Fetch more items
const loadMore = useCallback(async () => {
// Prevent concurrent requests
if (state.isLoading || pendingRequestRef.current || !state.hasMore) {
return;
}
setState(prev => ({ ...prev, isLoading: true, error: null }));
abortControllerRef.current = new AbortController();
try {
const request = fetchItems(state.cursor);
pendingRequestRef.current = request;
const result = await request;
// Check if request was aborted
if (abortControllerRef.current.signal.aborted) {
return;
}
// Prevent duplicates: filter out items already in state
const existingIds = new Set(state.items.map(item => item.id));
const newItems = result.items.filter(
item => !existingIds.has(item.id)
);
setState(prev => {
const combined = [...prev.items, ...newItems];
// Trim old items if exceeding maxItems
const trimmed = combined.length > maxItems
? combined.slice(combined.length - maxItems)
: combined;
return {
...prev,
items: trimmed,
cursor: result.nextCursor,
hasMore: result.hasMore !== false,
isLoading: false,
error: null,
};
});
pendingRequestRef.current = null;
} catch (error) {
if (error instanceof Error && error.name === 'AbortError') {
return; // Request was cancelled
}
const err = error instanceof Error ? error : new Error('Unknown error');
setState(prev => ({
...prev,
isLoading: false,
error: err,
}));
pendingRequestRef.current = null;
}
}, [state.cursor, state.hasMore, state.isLoading, state.items, fetchItems, maxItems]);
// Retry failed request
const retry = useCallback(() => {
setState(prev => ({ ...prev, error: null }));
loadMore();
}, [loadMore]);
// Reset to initial state
const reset = useCallback(() => {
if (abortControllerRef.current) {
abortControllerRef.current.abort();
}
setState({
items: [],
isLoading: false,
error: null,
hasMore: true,
cursor: initialCursor,
});
pendingRequestRef.current = null;
}, [initialCursor]);
// Detect when user scrolls near bottom
useEffect(() => {
if (typeof window === 'undefined') {
return;
}
const handleScroll = () => {
const scrollTop = window.scrollY || document.documentElement.scrollTop;
const scrollHeight = document.documentElement.scrollHeight;
const clientHeight = window.innerHeight;
const isNearBottom = scrollHeight - (scrollTop + clientHeight) < threshold;
if (isNearBottom && state.hasMore && !state.isLoading) {
loadMore();
}
};
window.addEventListener('scroll', handleScroll, { passive: true });
return () => {
window.removeEventListener('scroll', handleScroll);
};
}, [state.hasMore, state.isLoading, loadMore, threshold]);
// Cleanup abort controller on unmount
useEffect(() => {
return () => {
if (abortControllerRef.current) {
abortControllerRef.current.abort();
}
};
}, []);
return {
...state,
retry,
reset,
};
}
JavaScript Version
export function useInfiniteScroll(options) {
const {
fetchItems,
initialCursor,
threshold = 500,
maxItems = 1000,
} = options;
const [state, setState] = useState({
items: [],
isLoading: false,
error: null,
hasMore: true,
cursor: initialCursor,
});
const pendingRequestRef = useRef(null);
const abortControllerRef = useRef(null);
const loadMore = useCallback(async () => {
if (state.isLoading || pendingRequestRef.current || !state.hasMore) {
return;
}
setState(prev => ({ ...prev, isLoading: true, error: null }));
abortControllerRef.current = new AbortController();
try {
const request = fetchItems(state.cursor);
pendingRequestRef.current = request;
const result = await request;
if (abortControllerRef.current.signal.aborted) {
return;
}
const existingIds = new Set(state.items.map(item => item.id));
const newItems = result.items.filter(
item => !existingIds.has(item.id)
);
setState(prev => {
const combined = [...prev.items, ...newItems];
const trimmed = combined.length > maxItems
? combined.slice(combined.length - maxItems)
: combined;
return {
...prev,
items: trimmed,
cursor: result.nextCursor,
hasMore: result.hasMore !== false,
isLoading: false,
error: null,
};
});
pendingRequestRef.current = null;
} catch (error) {
if (error instanceof Error && error.name === 'AbortError') {
return;
}
const err = error instanceof Error ? error : new Error('Unknown error');
setState(prev => ({
...prev,
isLoading: false,
error: err,
}));
pendingRequestRef.current = null;
}
}, [state.cursor, state.hasMore, state.isLoading, state.items, fetchItems, maxItems]);
const retry = useCallback(() => {
setState(prev => ({ ...prev, error: null }));
loadMore();
}, [loadMore]);
const reset = useCallback(() => {
if (abortControllerRef.current) {
abortControllerRef.current.abort();
}
setState({
items: [],
isLoading: false,
error: null,
hasMore: true,
cursor: initialCursor,
});
pendingRequestRef.current = null;
}, [initialCursor]);
useEffect(() => {
if (typeof window === 'undefined') return;
const handleScroll = () => {
const scrollTop = window.scrollY || document.documentElement.scrollTop;
const scrollHeight = document.documentElement.scrollHeight;
const clientHeight = window.innerHeight;
const isNearBottom = scrollHeight - (scrollTop + clientHeight) < threshold;
if (isNearBottom && state.hasMore && !state.isLoading) {
loadMore();
}
};
window.addEventListener('scroll', handleScroll, { passive: true });
return () => {
window.removeEventListener('scroll', handleScroll);
};
}, [state.hasMore, state.isLoading, loadMore, threshold]);
useEffect(() => {
return () => {
if (abortControllerRef.current) {
abortControllerRef.current.abort();
}
};
}, []);
return {
...state,
retry,
reset,
};
}
Advanced Patterns: Retries and Error Recovery
Exponential Backoff Retry
For transient network failures, retry with exponential backoff:
interface UseInfiniteScrollAdvancedOptions<T>
extends UseInfiniteScrollOptions<T> {
// Max retry attempts for failed requests
maxRetries?: number;
// Base delay for exponential backoff (ms)
retryDelay?: number;
// Only retry on specific status codes
retryableStatusCodes?: number[];
}
export function useInfiniteScrollWithRetry<T extends { id: string | number }>(
options: UseInfiniteScrollAdvancedOptions<T>
) {
const {
maxRetries = 3,
retryDelay = 1000,
retryableStatusCodes = [408, 429, 500, 502, 503, 504],
...baseOptions
} = options;
const retryCountRef = useRef(0);
const fetchWithRetry = useCallback(
async (cursor?: string | number) => {
try {
return await options.fetchItems(cursor);
} catch (error) {
const isRetryable =
retryableStatusCodes.includes((error as any).status) ||
!(error instanceof Error && error.message.includes('400'));
if (!isRetryable || retryCountRef.current >= maxRetries) {
throw error;
}
retryCountRef.current++;
const delayMs = retryDelay * Math.pow(2, retryCountRef.current - 1);
// Wait before retrying
await new Promise(resolve => setTimeout(resolve, delayMs));
// Retry
return fetchWithRetry(cursor);
}
},
[options, maxRetries, retryDelay, retryableStatusCodes]
);
return useInfiniteScroll({
...baseOptions,
fetchItems: fetchWithRetry,
});
}
JavaScript Version
export function useInfiniteScrollWithRetry(options) {
const {
maxRetries = 3,
retryDelay = 1000,
retryableStatusCodes = [408, 429, 500, 502, 503, 504],
...baseOptions
} = options;
const retryCountRef = useRef(0);
const fetchWithRetry = useCallback(
async (cursor) => {
try {
return await options.fetchItems(cursor);
} catch (error) {
const isRetryable =
retryableStatusCodes.includes(error.status) ||
!(error instanceof Error && error.message.includes('400'));
if (!isRetryable || retryCountRef.current >= maxRetries) {
throw error;
}
retryCountRef.current++;
const delayMs = retryDelay * Math.pow(2, retryCountRef.current - 1);
await new Promise(resolve => setTimeout(resolve, delayMs));
return fetchWithRetry(cursor);
}
},
[options, maxRetries, retryDelay, retryableStatusCodes]
);
return useInfiniteScroll({
...baseOptions,
fetchItems: fetchWithRetry,
});
}
Intersection Observer Integration
For better performance with large lists, use Intersection Observer instead of scroll events:
interface UseInfiniteScrollIOOptions<T>
extends UseInfiniteScrollOptions<T> {
// Root margin for trigger (similar to CSS)
rootMargin?: string;
}
export function useInfiniteScrollIO<T extends { id: string | number }>(
options: UseInfiniteScrollIOOptions<T>
): InfiniteScrollState<T> & {
sentinelRef: React.RefObject<HTMLDivElement>;
retry: () => void;
reset: () => void;
} {
const { rootMargin = '500px', ...baseOptions } = options;
const baseScroll = useInfiniteScroll(baseOptions);
const sentinelRef = useRef<HTMLDivElement>(null);
useEffect(() => {
const observer = new IntersectionObserver(
(entries) => {
entries.forEach(entry => {
// Trigger load when sentinel becomes visible
if (entry.isIntersecting && baseScroll.hasMore && !baseScroll.isLoading) {
// Call the loadMore function from base hook
// Note: This requires exposing loadMore from base hook
}
});
},
{ rootMargin }
);
if (sentinelRef.current) {
observer.observe(sentinelRef.current);
}
return () => {
observer.disconnect();
};
}, [baseScroll.hasMore, baseScroll.isLoading]);
return {
...baseScroll,
sentinelRef,
};
}
Cursor-Based vs Offset-Based Pagination
Offset-Based (Page Number)
interface OffsetPaginationParams {
limit: number;
offset: number;
}
export function useOffsetPaginatedScroll<T extends { id: string | number }>(
fetchFn: (params: OffsetPaginationParams) => Promise<T[]>
) {
const limit = 20;
return useInfiniteScroll({
fetchItems: async (offset: number = 0) => {
const items = await fetchFn({ limit, offset });
return {
items,
nextCursor: offset + limit,
hasMore: items.length === limit,
};
},
initialCursor: 0,
});
}
Cursor-Based (Stable Pagination)
Cursor-based pagination is more stable for real-time data (feeds):
interface CursorPaginationParams {
cursor?: string;
limit: number;
}
export function useCursorPaginatedScroll<T extends { id: string | number }>(
fetchFn: (params: CursorPaginationParams) => Promise<{
items: T[];
nextCursor?: string;
}>
) {
const limit = 20;
return useInfiniteScroll({
fetchItems: async (cursor?: string) => {
const result = await fetchFn({ cursor, limit });
return {
items: result.items,
nextCursor: result.nextCursor,
hasMore: result.items.length === limit && !!result.nextCursor,
};
},
});
}
Practical Application Scenarios
Scenario 1: Social Media Feed with Optimistic Updates
interface Post {
id: string;
author: string;
content: string;
timestamp: number;
}
export function SocialFeed() {
const { items, isLoading, error, hasMore, retry } = useInfiniteScroll<Post>({
fetchItems: async (cursor) => {
const response = await fetch(
`/api/posts?cursor=${cursor || ''}&limit=20`
);
if (!response.ok) {
throw new Error(`Failed to fetch posts: ${response.status}`);
}
const data = await response.json();
return {
items: data.posts,
nextCursor: data.nextCursor,
hasMore: data.hasMore,
};
},
threshold: 1000, // Load before reaching very bottom
});
return (
<div className="feed">
{items.map(post => (
<article key={post.id} className="post">
<h3>{post.author}</h3>
<p>{post.content}</p>
<time>{new Date(post.timestamp).toLocaleString()}</time>
</article>
))}
{isLoading && (
<div className="loading-indicator">
<div className="spinner">⟳</div> Loading more posts...
</div>
)}
{error && (
<div className="error-message">
<p>Failed to load posts: {error.message}</p>
<button onClick={retry}>Retry</button>
</div>
)}
{!hasMore && items.length > 0 && (
<div className="end-message">No more posts to load</div>
)}
{items.length === 0 && !isLoading && (
<div className="empty-state">No posts yet</div>
)}
</div>
);
}
Scenario 2: E-commerce Product Grid with Filtering
interface Product {
id: string;
name: string;
price: number;
image: string;
}
export function ProductGrid({ category }: { category: string }) {
const [filter, setFilter] = useState('');
const { items, isLoading, error, reset } = useInfiniteScroll<Product>({
fetchItems: async (cursor) => {
const params = new URLSearchParams({
category,
filter,
cursor: cursor?.toString() || '',
limit: '30',
});
const response = await fetch(`/api/products?${params}`);
const data = await response.json();
return {
items: data.products,
nextCursor: data.cursor,
hasMore: data.hasMore,
};
},
});
// Reset when filter changes
useEffect(() => {
reset();
}, [filter, category, reset]);
return (
<div>
<input
type="text"
placeholder="Search products..."
value={filter}
onChange={e => setFilter(e.target.value)}
/>
<div className="product-grid">
{items.map(product => (
<div key={product.id} className="product-card">
<img src={product.image} alt={product.name} />
<h3>{product.name}</h3>
<p className="price">${product.price.toFixed(2)}</p>
</div>
))}
</div>
{isLoading && <div>Loading more products...</div>}
{error && <div>Error: {error.message}</div>}
</div>
);
}
Scenario 3: Comment Thread with Lazy Loading
interface Comment {
id: string;
author: string;
text: string;
replies?: Comment[];
repliesCount: number;
}
export function CommentThread({ postId }: { postId: string }) {
const { items: comments, isLoading, error } = useInfiniteScroll<Comment>({
fetchItems: async (cursor) => {
const response = await fetch(
`/api/posts/${postId}/comments?cursor=${cursor || ''}&limit=50&sort=latest`
);
const data = await response.json();
return {
items: data.comments,
nextCursor: data.nextCursor,
hasMore: data.hasMore,
};
},
maxItems: 500, // Keep recent comments in memory
});
return (
<div className="comments">
<h2>Comments ({comments.length})</h2>
{comments.map(comment => (
<div key={comment.id} className="comment">
<div className="comment-header">
<strong>{comment.author}</strong>
</div>
<p>{comment.text}</p>
{comment.repliesCount > 0 && (
<button className="show-replies">
Show {comment.repliesCount} replies
</button>
)}
</div>
))}
{isLoading && <div>Loading comments...</div>}
{error && <div>Error: {error.message}</div>}
</div>
);
}
Performance and Scaling
Virtual Scrolling for Large Lists
For thousands of items, use virtualization to render only visible items:
import { FixedSizeList as List } from 'react-window';
interface VirtualizedInfiniteListProps<T> {
items: T[];
isLoading: boolean;
itemSize: number;
renderItem: (item: T, index: number) => ReactNode;
}
export function VirtualizedInfiniteList<T extends { id: string | number }>({
items,
isLoading,
itemSize,
renderItem,
}: VirtualizedInfiniteListProps<T>) {
const listRef = useRef<List>(null);
const itemCount = isLoading ? items.length + 1 : items.length;
const Row = ({ index, style }: { index: number; style: CSSProperties }) => {
if (index >= items.length) {
return <div style={style}>Loading...</div>;
}
return (
<div style={style}>
{renderItem(items[index], index)}
</div>
);
};
return (
<List
ref={listRef}
height={600}
itemCount={itemCount}
itemSize={itemSize}
width="100%"
>
{Row}
</List>
);
}
Caching and Cache Invalidation
const cacheRef = useRef<Map<string, any>>(new Map());
export function useCachedInfiniteScroll<T extends { id: string | number }>(
cacheKey: string,
options: UseInfiniteScrollOptions<T>
) {
return useInfiniteScroll({
...options,
fetchItems: async (cursor) => {
const key = `${cacheKey}:${cursor}`;
// Return cached result if available
if (cacheRef.current.has(key)) {
return cacheRef.current.get(key);
}
// Fetch and cache
const result = await options.fetchItems(cursor);
cacheRef.current.set(key, result);
return result;
},
});
}
// Invalidate cache when needed
export function invalidateCache(pattern: string) {
const keysToDelete = Array.from(cacheRef.current.keys()).filter(key =>
key.startsWith(pattern)
);
keysToDelete.forEach(key => cacheRef.current.delete(key));
}
FAQ
Q: How do I prevent duplicate items?
A: The hook filters items by ID before adding them:
const existingIds = new Set(state.items.map(item => item.id));
const newItems = result.items.filter(item => !existingIds.has(item.id));
This handles cases where items appear in multiple pages (common with real-time data).
Q: Should I use offset-based or cursor-based pagination?
A: Cursor-based is superior for most real-time applications:
- Offset-based: Simple but breaks with insertions/deletions (user sees duplicates or misses items)
- Cursor-based: Stable for real-time data, handles concurrent changes, scales better
Use offset-based only for static, historical data.
Q: How do I handle the "pull to refresh" pattern?
A: Use the reset() method returned by the hook:
const { items, reset } = useInfiniteScroll(options);
const handlePullToRefresh = () => {
reset(); // Clears items and re-fetches from beginning
};
Q: What if the server returns items in different order on subsequent pages?
A: This indicates a data consistency problem. Cursor-based pagination prevents this. If using offset-based, ensure the sorting is stable (consistent across pages).
Q: How do I implement "load more" button instead of automatic scroll-based loading?
A: Create a variant that doesn't attach scroll listeners:
export function useManualInfiniteScroll<T extends { id: string | number }>(
options: UseInfiniteScrollOptions<T>
) {
const scroll = useInfiniteScroll({
...options,
threshold: Infinity, // Never auto-trigger
});
return {
...scroll,
loadMore: () => scroll.loadMore?.(), // Expose load function
};
}
Q: How do I implement bidirectional scroll (load older and newer)?
A: Maintain separate cursor for each direction:
interface BidirectionalState<T> {
items: T[];
newerCursor?: string;
olderCursor?: string;
isLoadingNewer: boolean;
isLoadingOlder: boolean;
}
export function useBidirectionalScroll<T extends { id: string | number }>(
options: UseInfiniteScrollOptions<T>
): BidirectionalState<T> {
// Track separate cursors for new/old content
// Detect scroll direction and load accordingly
// Return combined state
}
Q: How do I handle items being deleted while scrolling?
A: The hook's deduplication logic handles this:
// If an item was deleted, it simply won't appear in next fetch
const existingIds = new Set(state.items.map(item => item.id));
const newItems = result.items.filter(item => !existingIds.has(item.id));
Q: What's the memory impact of maxItems?
A: With maxItems: 1000 and average item size of 500 bytes, you'd use ~500KB. This is safe for modern browsers. Adjust based on your item size and target devices.
Common Patterns
Pattern 1: Search Results with Pagination
function SearchResults({ query }: { query: string }) {
const { items, isLoading, reset } = useInfiniteScroll({
fetchItems: async (cursor) => {
const results = await searchAPI(query, cursor);
return {
items: results.hits,
nextCursor: results.nextCursor,
hasMore: results.hasMore,
};
},
});
useEffect(() => {
reset(); // Reset when query changes
}, [query, reset]);
return (
<div>
{items.map(result => (
<SearchResult key={result.id} result={result} />
))}
</div>
);
}
Pattern 2: User Timeline with Real-Time Updates
function Timeline() {
const { items, isLoading } = useInfiniteScroll({
fetchItems: async (cursor) => {
// Cursor is timestamp of last item
const posts = await fetchUserTimeline(cursor);
return {
items: posts,
nextCursor: posts[posts.length - 1]?.timestamp,
hasMore: posts.length === 20,
};
},
});
// Prepend real-time updates (websocket)
useEffect(() => {
const unsubscribe = socketIO.on('newPost', (post) => {
// Insert at beginning, deduplication handles it if it reappears
setItems(prev => [post, ...prev]);
});
return unsubscribe;
}, []);
}
Related Articles
- useScrollPosition Hook: Tracking Scroll State
- Intersection Observer Pattern
- Virtualization for Large Lists
- Error Handling and Retry Logic
Next Steps
The useInfiniteScroll hook scales to handle massive datasets when combined with:
- Virtual scrolling for rendering only visible items
- Cache strategies for offline-first apps
- Real-time updates via WebSocket or Server-Sent Events
- Pagination strategies (offset vs cursor)
- Analytics tracking scroll depth
At scale (ByteDance feed with millions of daily users), infinite scroll becomes critical infrastructure. Master the patterns here and you can build feeds that handle production traffic.
What scaling challenges do you face with infinite scroll? Share your implementations in the comments—bidirectional loading, real-time updates, and cache strategies at scale always generate interesting discussions.
Google AdSense Placeholder
CONTENT SLOT