Skip to main content
This guide will walk you through downloading your first web novel with Universal Novel Scraper. By the end, you’ll have a professionally formatted EPUB ready to read on any device.

​
Overview

UNS uses a browser-based scraping approach that mimics human behavior, allowing it to:
  • Bypass bot detection systems like Cloudflare
  • Handle dynamic JavaScript-heavy websites
  • Maintain consistent scraping sessions
  • Extract clean chapter content automatically
Make sure you’ve completed the Installation steps before continuing.

​
Your First Download

1

Launch UNS and Navigate to Download Manager

Open the Universal Novel Scraper application. You’ll see the main interface with three tabs in the sidebar:
  • Download - Active downloads and job controls
  • History - Resume or delete previous downloads
  • Library - View and manage your EPUB collection
Click on the Download tab if not already selected.
Download Manager Interface
2

Choose a Provider

UNS uses provider scripts to support different novel websites. The app comes with several built-in providers.To browse available sources:
  1. Click the Providers button in the toolbar
  2. View installed provider scripts
  3. Install additional providers from the community repository if needed
Popular providers include sites like NovelUpdates, ReadLightNovel, and WebNovel. Install providers that host your favorite novels.
Provider Marketplace
3

Search for a Novel

Navigate to the Search tab to find novels across all installed providers:
  1. Enter the novel title in the search bar
  2. Select which providers to search (or search all)
  3. Click Search
The search aggregates results from multiple sources in real-time.
Multi-Provider Search
You can also browse trending and recently updated novels by category:
  1. Select a provider from the dropdown
  2. Choose a category (Action, Romance, Fantasy, etc.)
  3. Browse paginated results
Category Browsing
4

Select Novel and Starting Chapter

Click on a novel from search results to view its details:
  • Description - Plot synopsis
  • Author - Creator information
  • Chapter List - All available chapters
To start downloading:
  1. Select your starting chapter from the dropdown
  2. Optionally customize the novel metadata (title, author)
  3. Upload a custom cover image if desired (or use the default)
  4. Click Start Download
UNS will automatically scrape all chapters starting from your selected chapter until it reaches the last available chapter or a dead-end link.
5

Monitor Download Progress

Once started, the Download Manager shows real-time progress:
  • Status Messages - Current action (Loading, Saving, Cloudflare detected)
  • Chapter Count - Number of chapters saved
  • Live Console - Detailed scraping logs
  • Browser Window Toggle - View the live scraping session
// Example console output
Chapter 1: Fetching...
✓ Saved Chapter 1
Chapter 2: Fetching...
✓ Saved Chapter 2
Chapter 3: Fetching...
Click “Show Browser” to watch the scraper navigate pages in real-time. This is helpful for debugging or verifying the scraping process.
6

Handle Cloudflare Challenges (If Needed)

Some websites use Cloudflare protection. If detected, UNS can handle it:
  1. Enable “Cloudflare Bypass Mode” before starting the download
  2. If a challenge appears, the browser window will automatically show
  3. Manually solve the CAPTCHA or verification
  4. Scraping resumes automatically once cleared
With Cloudflare bypass enabled, UNS adds random delays (1.5-4 seconds) between chapters to appear more human-like. Downloads will take longer but are more reliable.
// Cloudflare detection code from main.js:126
async function detectCloudflare(window) {
  const title = await window.webContents.getTitle();
  const titleIndicators = ['just a moment', 'cloudflare', 
    'attention required', 'verify you are human'];
  return titleIndicators.some(i => title.toLowerCase().includes(i));
}
7

Download Completes

When all chapters are scraped, UNS automatically:
  1. Compiles chapters into a single EPUB file
  2. Embeds metadata (title, author)
  3. Adds the cover image
  4. Saves to your library
You’ll see:
📦 Generating EPUB...
✅ Success!
The EPUB is now available in your Library.

​
Accessing Your EPUB

1

Navigate to Library

Click the Library tab in the sidebar. You’ll see all your downloaded novels with:
  • Cover Images - Automatically extracted from EPUBs
  • Title & Author - From embedded metadata
  • Action Buttons - Open, Export, or Delete
Library Grid View
2

Read or Export

You have several options:Open in UNS Reader
  • Click the book cover to open the built-in EPUB reader
  • Navigate with arrow keys or click navigation
  • Bookmark and track reading progress
Built-in EPUB Reader
Export to External Reader
  • Click the Export button to download the EPUB file
  • Transfer to your e-reader (Kindle, Kobo, etc.)
  • Open in apps like Calibre, Apple Books, or Google Play Books
Open Output Folder
  • Click “Open Folder” to access the storage directory
  • Files are stored in output/epubs/ in your user data folder

​
Resuming Interrupted Downloads

If a download is interrupted (app crash, network issue, manual stop), you can resume from where it left off:
1

Open History Tab

Navigate to the History tab. You’ll see all incomplete downloads with:
  • Novel name
  • Current progress (“X chapters scraped”)
  • Status (Processing, Paused, Error)
Download History
2

Resume Download

Click the Resume button next to the paused download. UNS will:
  1. Load the saved job state from backend/output/jobs/<job_id>_progress.jsonl
  2. Read the last successfully scraped chapter
  3. Continue from the next chapter URL
  4. Preserve all previously downloaded content
# Resume logic from backend/api.py:244
@app.get("/api/status/{job_id}")
def check_status(job_id: str):
    progress_file = get_progress_file(job_id)
    if os.path.exists(progress_file):
        with open(progress_file, "r", encoding="utf-8") as f:
            chapter_count = sum(1 for _ in f)
    return {"chapters_count": chapter_count, ...}
Do not delete items from History until the EPUB is fully generated. Deleting removes all temporary chapter data, and you’ll need to restart the download from scratch.

​
Tips for Successful Downloads

  • Disable Cloudflare bypass for sites that don’t require it (faster scraping)
  • Close the browser window when not needed (reduces resource usage)
  • Download during off-peak hours to avoid rate limiting
If a site consistently fails:
  1. Enable Cloudflare bypass mode and add random delays
  2. Use the browser window to check for additional verification steps
  3. Check provider scripts - they may need updating for site changes
  4. Try a different provider - the same novel may be on multiple sites
  • Upload a high-quality cover before starting (recommended 600x800px or larger)
  • Verify author and title are correct (hard to change after generation)
  • Check the first few chapters to ensure content extraction is working properly
EPUBs can get large for long novels (500+ chapters):
  • Export and delete finished novels you’ve already transferred to an e-reader
  • Clear history for completed downloads to remove temporary files
  • Monitor the output folder periodically
Storage locations:
  • Windows: C:\Users\<You>\AppData\Roaming\UNS\output\epubs
  • macOS: ~/Library/Application Support/UNS/output/epubs
  • Linux: ~/.config/UNS/output/epubs

​
Understanding the Scraping Process

UNS uses a sophisticated multi-stage scraping architecture:

​
Architecture Overview

┌─────────────────┐
│   React UI      │  User initiates download
└────────┬────────┘
         │ IPC
         ▼
┌─────────────────┐
│ Electron Main   │  Manages browser session
└────────┬────────┘
         │ Controls
         ▼
┌─────────────────┐
│ Chromium Window │  Loads pages like a human
└────────┬────────┘
         │ Extracts HTML
         ▼
┌─────────────────┐
│ Provider Script │  Parses content with custom rules
└────────┬────────┘
         │ POST /api/save-chapter
         ▼
┌─────────────────┐
│ FastAPI Backend │  Stores chapters, generates EPUB
└─────────────────┘

​
How Chapter Extraction Works

Provider scripts inject custom JavaScript into each page: 
// Example from main.js:180
pageData = await scraperWindow.webContents.executeJavaScript(`
  (() => {
    // Extract chapter title
    const title = document.querySelector('.chapter-title, h1')?.innerText;
    
    // Extract content paragraphs
    const paragraphs = Array.from(
      document.querySelectorAll('.chapter-content p')
    ).map(p => p.innerText.trim()).filter(p => p.length > 0);
    
    // Find "Next Chapter" link
    const nextBtn = Array.from(document.querySelectorAll('a')).find(a => {
      return (a.innerText || '').toLowerCase().includes('next');
    });
    
    return { title, paragraphs, nextUrl: nextBtn?.href };
  })()
`);
This extracted data is then sent to the Python backend for storage.

​
Common Issues and Solutions

This means the provider script couldn’t locate chapter text on the page.Solutions:
  1. Check if the website structure changed (inspect the page)
  2. Update the provider script from the marketplace
  3. Report the issue to the provider maintainer
  4. Try an alternative source for the same novel
The scraper couldn’t find a “Next Chapter” link.Causes:
  • The link uses javascript:; instead of a real URL
  • The link is hidden behind a login wall
  • You’ve reached the last available chapter
Solution: Manually navigate to the chapter list and resume from a specific chapter URL.
This happens if the “Next Chapter” link points to the same page.UNS has built-in duplicate detection:
// From main.js:196
absoluteHref.split('#')[0] !== window.location.href.split('#')[0]
If duplicates still occur, the provider script may need updating.
Verify the EPUB is valid:
  1. Try opening in multiple readers (Calibre is most lenient)
  2. Check file size - 0 KB indicates generation failed
  3. Look at console logs for Python errors during finalize-epub
Common causes:
  • Invalid cover image format
  • Corrupt chapter data
  • Insufficient disk space

​
Next Steps

Now that you’ve downloaded your first novel, explore advanced features:

Search & Discovery

Master multi-provider search and category browsing

Provider System

Install and create custom website scrapers

Library Management

Organize, read, and export your collection

Cloudflare Bypass

Handle protected sites with manual control
Having trouble? Check the Troubleshooting page or open an issue on GitHub.