Web Unlocker is a powerful web content retrieval service that supports complex web page rendering and interaction scenarios.Basic Request Structure#
{
"actor": "unlocker.webunlocker",
"input": {
"url": "https://example.com",
"js_render": false,
"headless": false
},
"proxy": {
"country": "US"
}
}
Core Features#
JavaScript Rendering#
JavaScript rendering enables handling of dynamically loaded content and SPAs (Single Page Applications). Enables a complete browser environment, supporting more complex page interactions and rendering requirements.js_render=true
,we will use the browser to request.{
"actor": "unlocker.webunlocker",
"input": {
"url": "https://www.google.com/",
"js_render": true
},
"proxy": {
"country": "US"
}
}
JavaScript Instructions#
Provides an extensive set of JavaScript directives that allow you to dynamically interact with web pages.These directives enable you to click elements, fill out forms, submit forms, or wait for specific elements to appear, providing flexibility for tasks such as clicking a "read more" button or submitting a form.{
"actor": "unlocker.webunlocker",
"input": {
"url": "https://example.com",
"js_render": true,
"js_instructions": [
{
"wait_for": [
".dynamic-content",
30000
]
},
{
"click": [
"#load-more",
1000
]
},
{
"fill": [
"#search-input",
"search term"
]
},
{
"keyboard": [
"press",
"Enter"
]
},
{
"evaluate": "window.scrollTo(0, document.body.scrollHeight)"
}
]
}
}
Here are some common actions you can perform with JavaScript Instructions:JavaScript Instructions Reference:Instruction | Syntax | Description | Example |
---|
wait_for | [selector, timeout] | Wait for element to appear | {"wait_for": [".content", 30000]} |
click | [selector, delay] | Click element | {"click": [".button", 1000]} |
fill | [selector, value] | Fill form | {"fill": ["#input", "text"]} |
wait | milliseconds | Fixed wait time | {"wait": 2000} |
evaluate | javascript_code | Execute JS code | {"evaluate": "console.log('test')"} |
keyboard | [action, value, delay?] | Keyboard operation | See keyboard operations table below |
Operation | Syntax | Description | Example |
---|
Press key | ["press", keyInput] | Press a specific keyInput | {"keyboard": ["press", "Enter"]} |
Type text | ["type", text, delay?] | Type text with optional delay | {"keyboard": ["type", "Hello", 20]} |
Key down | ["down", key] | Hold down a key | {"keyboard": ["down", "Shift"]} |
Key up | ["up", key] | Release a key | {"keyboard": ["up", "Shift"]} |
Resource Control#
Resource loading control system for optimizing performance and bandwidth usage.{
"actor": "unlocker.webunlocker",
"input": {
"url": "https://example.com",
"js_render": true,
"block": {
"resources": [
"Image",
"Font",
"Stylesheet",
"Script"
],
"urls": [
"*.analytics.com/*",
"*/ads/*"
]
}
}
}
Complete Resource Types Reference:Resource Type | Description | Impact |
---|
Document | Main document and iframes | Core page content |
Stylesheet | CSS files | Page styling and layout |
Image | Images and icons | Visual content |
Media | Audio and video resources | Multimedia content |
Font | Web fonts | Text rendering |
Script | JavaScript files | Page functionality |
TextTrack | Video subtitles and captions | Media accessibility |
XHR | XMLHttpRequest calls | Legacy async requests |
Fetch | Fetch API requests | Modern async requests |
Prefetch | Prefetched resources | Performance optimization |
EventSource | Server-sent events | Real-time updates |
WebSocket | WebSocket connections | Bidirectional communication |
Manifest | Web app manifests | PWA configuration |
SignedExchange | Signed HTTP exchanges | Content authenticity |
Ping | Ping requests | Analytics and tracking |
CSPViolationReport | CSP violation reports | Security monitoring |
Preflight | CORS preflight requests | Cross-origin security |
Other | Unclassified resources | Miscellaneous |
{
"actor": "unlocker.webunlocker",
"input": {
"url": "https://example.com",
"js_render": true,
"block": {
"resources": [
"Image",
"Font",
"Stylesheet",
"Script",
"Media",
"Ping",
"Prefetch"
]
}
}
}
Best Practices for Resource Blocking:1.
Enable js_render
only when necessary
Use resource blocking wisely, Block non-essential resources for faster loading
Consider blocking Prefetch
and Ping
for reduced network usage
Keep Document
and critical Script
resources unblocked
2.
Block Image
and Media
for bandwidth-intensive pages
Consider blocking Font
to use system fonts instead
3.
Implement request retry mechanisms
Use wait_for
instead of fixed wait
4.
Close unnecessary connections promptly
Note: Resource type strings are case-sensitive. Use exact matches as shown in the reference table. Modified at 2025-01-07 10:51:01