WKZombie

<img align="left" src="https://raw.githubusercontent.com/mkoehnke/WKZombie/master/Resources/Documentation/Logo.png" hspace="30" width="140px">

WKZombie is an iOS/OSX web-browser without a graphical user interface. It was developed as an experiment in order to familiarize myself with using functional concepts written in Swift 4.

It incorporates WebKit (WKWebView) for rendering and hpple (libxml2) for parsing the HTML content. In addition, it can take snapshots and has rudimentary support for parsing/decoding JSON elements. Chaining asynchronous actions makes the code compact and easy to use.

Use Cases

There are many use cases for a Headless Browser. Some of them are:

• Collect data without an API
• Scraping websites
• Automating interaction of websites
• Manipulation of websites
• Running automated tests / snapshots
• etc.

Example

The following example is supposed to demonstrate the WKZombie functionality. Let's assume that we want to show all iOS Provisioning Profiles in the Apple Developer Portal.

Manual Web-Browser Navigation

When using a common web-browser (e.g. Mobile Safari) on iOS, you would typically type in your credentials, sign in and navigate (via links) to the Provisioning Profiles section:

<img src="https://raw.githubusercontent.com/mkoehnke/WKZombie/master/Resources/Documentation/WKZombie-Web-Demo.gif" />

Automation with WKZombie

The same navigation process can be reproduced automatically within an iOS/OSX app linking WKZombie Actions. In addition, it is now possible to manipulate or display this data in a native way with UITextfield, UIButton and a UITableView.

<img src="https://raw.githubusercontent.com/mkoehnke/WKZombie/master/Resources/Documentation/WKZombie-Simulator-Demo.gif" />

Take a look at the iOS/OSX demos in the Example directory to see how to use it.

Getting Started

iOS / OSX

The best way to get started is to look at the sample project. Just run the following commands in your shell and you're good to go:

$ cd Example
$ pod install
$ open Example.xcworkspace

Note: You will need CocoaPods 1.0 beta4 or higher.

Command-Line

For a Command-Line demo, run the following commands inside the WKZombie root folder:

$ swift build -Xcc -I/usr/include/libxml2 -Xlinker -lxml2

$ .build/debug/Example <appleid> <password>

Usage

A WKZombie instance equates to a web session. Top-level convenience methods like WKZombie.open() use a shared instance, which is configured with the default settings.

As such, the following three statements are equivalent:

let action : Action<HTMLPage> = open(url)

let action : Action<HTMLPage> = WKZombie.open(url)

let browser = WKZombie.sharedInstance
let action : Action<HTMLPage> = browser.open(url)

Applications can also create their own WKZombie instance:

self.browser = WKZombie(name: "Demo")

Be sure to keep browser in a stored property for the time of being used.

a. Chaining Actions

Web page navigation is based on Actions, that can be executed implicitly when chaining actions using the >>> or >>* (for snapshots) operators. All chained actions pass their result to the next action. The === operator then starts the execution of the action chain.

The following snippet demonstrates how you would use WKZombie to collect all Provisioning Profiles from the Developer Portal and take snapshots of every page:

    open(url)
>>* get(by: .id("accountname"))
>>> setAttribute("value", value: user)
>>* get(by: .id("accountpassword"))
>>> setAttribute("value", value: password)
>>* get(by: .name("form2"))
>>> submit
>>* get(by: .contains("href", "/account/"))
>>> click(then: .wait(2.5))
>>* getAll(by: .contains("class", "row-"))
=== myOutput

In order to output or process the collected data, one can either use a closure or implement a custom function taking the result as parameter:

func myOutput(result: [HTMLTableColumn]?) {
  // handle result
}

or

func myOutput(result: Result<[HTMLTableColumn]>) {
  switch result {
  case .success(let value): // handle success
  case .error(let error): // handle error
  }
}

b. Manual Actions

Actions can also be started manually by calling the start() method:

let action : Action<HTMLPage> = browser.open(url)

action.start { result in
    switch result {
    case .success(let page): // process page
    case .error(let error):  // handle error
    }
}

This is certainly the less complicated way, but you have to write a lot more code, which might become confusing when you want to execute Actions successively.

Basic Action Functions

There are currently a few Actions implemented, helping you visit and navigate within a website:

Open a Website

The returned WKZombie Action will load and return a HTML or JSON page for the specified URL.

func open<T : Page>(url: URL) -> Action<T>

Optionally, a PostAction can be passed. This is a special wait/validation action, that is performed after the page has finished loading. See PostAction for more information.

func open<T : Page>(then: PostAction) -> (url: URL) -> Action<T>

Get the current Website

The returned WKZombie Action will retrieve the current page.

func inspect<T: Page>() -> Action<T>

Submit a Form

The returned WKZombie Action will submit the specified HTML form.

func submit<T : Page>(form: HTMLForm) -> Action<T>

Optionally, a PostAction can be passed. See PostAction for more information.

func submit<T : Page>(then: PostAction) -> (form: HTMLForm) -> Action<T>

Click a Link / Press a Button

The returned WKZombie Actions will simulate the interaction with a HTML link/button.

func click<T: Page>(link : HTMLLink) -> Action<T>
func press<T: Page>(button : HTMLButton) -> Action<T>

Optionally, a PostAction can be passed. See [PostAction](#Special- Parameters) for more information.

func click<T: Page>(then: PostAction) -> (link : HTMLLink) -> Action<T>
func press<T: Page>(then: PostAction) -> (button : HTMLButton) -> Action<T>

Note: HTMLButton only works if the "onClick" HTML-Attribute is present. If you want to submit a HTML form, you should use Submit instead.

Find HTML Elements

The returned WKZombie Action will search the specified HTML page and return the first element matching the generic HTML element type and passed SearchType.

func get<T: HTMLElement>(by: SearchType<T>) -> (page: HTMLPage) -> Action<T>

The returned WKZombie Action will search and return all elements matching.

func getAll<T: HTMLElement>(by: SearchType<T>) -> (page: HTMLPage) -> Action<[T]>

Set an Attribute

The returned WKZombie Action will set or update an existing attribute/value pair on the specified HTMLElement.

func setAttribute<T: HTMLElement>(key: String, value: String?) -> (element: T) -> Action<HTMLPage>

Execute JavaScript

The returned WKZombie Actions will execute a JavaScript string.

func execute(script: JavaScript) -> (page: HTMLPage) -> Action<JavaScriptResult>
func execute(script: JavaScript) -> Action<JavaScriptResult>

For example, the following example shows how retrieve the title of the currently loaded website using the execute() method:

    browser.inspect
>>> browser.execute("document.title")
=== myOutput

func myOutput(result: JavaScriptResult?) {
  // handle result
}

The following code shows another way to execute JavaScript, that is e.g. value of an attribute:

    browser.open(url)
>>> browser.get(by: .id("div"))
>>> browser.map { $0.objectForKey("onClick")! }
>>> browser.execute
>>> browser.inspect
=== myOutput

func myOutput(result: HTMLPage?) {
  // handle result
}

Fetching

Some HTMLElements, that implement the HTMLFetchable protocol (e.g. HTMLLink or HTMLImage), contain attributes like "src" or "href", that link to remote objects or data. The following method returns a WKZombie Action that can conveniently download this data:

func fetch<T: HTMLFetchable>(fetchable: T) -> Action<T>

Once the fetch method has been executed, the data can be retrieved and converted. The following example shows how to convert data, fetched from a link, into an UIImage:

let image : UIImage? = link.fetchedContent()

Fetched data can be converted into types, that implement the HTMLFetchableContent protocol. The following types are currently supported:

• UIImage / NSImage
• Data

Note: See the OSX example for more info on how to use this.

Transform

The returned WKZombie Action will transform a WKZombie object into another object using the specified function f.

func map<T, A>(f: T -> A) -> (element: T) -> Action<A>

This functio