Hear See

Tell us about the features and technologies you used in your Swift playground.

Swift Playground Author Template: providing the general structure and development of the playground book.
ARKit: for measuring distances through recasting, scene reconstruction, and classification of surfaces
RealityKit: for displaying distance measurement, nearest point marker, and background/spatial music
RealityComposer: modeling the pin marker used for closest obstacle
SwiftUI: rendering additional UI elements, such as the button for changing the voice used for speaking, and the list of nearby point distances
AVFoundation (AVSpeechSynthesizer): announcing distances/other messages
StringInterpolation + NumberFormatter: custom interpolation allowing for formatting the distances to speak.

Comments

Is there anything else you would like us to know?

There’s a chance that, under unknown circumstances, the Playground live view will be interrupted during the RealityKit viewing experience. However, the LiveViewTestApp with identical code is confirmed to be working correctly without any issue. Examining the captured stack trace, we can see that the error originated from Metal, despite that not a single line of Metal code is written by me for this submission.

Confused, I reach out to others who are also working on the Swift Student Challenge and using RealityKit, and many have reported similar encounters. In the most extreme case, just adding a plane anchor to the ARView causes the remote process interruption, which shouldn’t happen given how simple the code is.

Therefore, I suspect there might be some internal issue with using RealityKit inside Swift Playgrounds. Nonetheless, this is not a severe issue, so re-running the code for the page after each interruption is highly recommended.

Thank you very much.

Playground Book Xcode Project

Overview

Welcome to the Playground Book Xcode project! This Xcode project is set up to produce two things:

A playground book
An app for debugging the live view

In support of this, there are five targets in this Xcode project:

PlaygroundBook: Produces a playground book as its output
BookCore: Compiles the BookCore auxiliary module, an auxiliary module that contains the implementation of the live view and any other author-only functionality
BookAPI: Compiles the BookAPI auxiliary module, an auxiliary module that is automatically imported into all user code throughout the book
UserModule: Compiles the UserModule user module, a blank user module where users may write code
LiveViewTestApp: Produces an app which uses the Book_Sources module to show the live view similarly to how it would be shown in Swift Playgrounds

This project includes the PlaygroundSupport and PlaygroundBluetooth frameworks from Swift Playgrounds to allow the BookCore, BookAPI, UserModule, and LiveViewTestApp targets to take full advantage of those APIs. The supporting content included with this template, including these frameworks, requires Xcode 12.2 to build. Attempting to use this template with another version of Xcode may result in build errors.

For more information about the playground book file format, see the Swift Playgrounds authoring documentation.

First Steps

To get started with this Xcode project, you need to make a few changes to personalize it for your playground book.

Open BuildSettings.xcconfig (in the “Config Files” group in the Project navigator) and make the following modifications:

Set BUNDLE_IDENTIFIER_PREFIX to a value appropriate for your team
Set PLAYGROUND_BOOK_FILE_NAME to a value appropriate for your playground book

You may also modify PLAYGROUND_BOOK_CONTENT_VERSION to set the ContentVersion in the book's Manifest.plist. PLAYGROUND_BOOK_CONTENT_IDENTIFIER may also be modified to customize to set a specific ContentIdentifier in the book's Manifest.plist. (It defaults to a value based on BUNDLE_IDENTIFIER_PREFIX and PLAYGROUND_BOOK_FILE_NAME.)

Open ManifestPlist.strings (in the “PrivateResources” group in the “PlaygroundBook” group in the Project navigator) and modify the values of the strings to be appropriate for your playground book.
Open the Project Editor, select the LiveViewTestApp target, and select the appropriate Team in the Signing section. (This step is not required if you are only testing with the iOS Simulator.)

Once you've finished configuring the project, you can build the PlaygroundBook target, which will produce your playground book as a product. (You can access it by opening the “Products” group in the Project navigator, and then right-clicking and selecting “Show in Finder”. From there, you can use AirDrop or other methods to copy the playground book to an iPad running Swift Playgrounds, or double-click it to import it into Swift Playgrounds on your Mac.)

Common Tasks

This Xcode project is structured both in Xcode and on-disk in a very particular way to ensure that the book is assembled correctly. Below are guides for accomplishing some common tasks when creating your playground book.

Adding a New Auxiliary or User Module

An auxiliary module is a Swift module which vends an API to the playground book without exposing the sources. A user module is a Swift module where the sources are fully editable by users. (For more information about auxiliary and user modules, see the Swift Playgrounds authoring documentation.)

To create an auxiliary module or user module, you need to create the right folder structure on-disk. You also need to create a static library target which builds the sources to get editor features such as code completion and live issues. This can be accomplished by performing the following steps:

In the Project navigator, expand the “PlaygroundBook” group
If you are adding an auxiliary module, expand the “Modules” group. If you are adding a user module, expand the “UserModules” group.
Control-click (or right-click) on the “Modules” or “UserModules” group
Choose “New Group” from the context menu
Name the newly-created group ModuleName.playgroundmodule, where “ModuleName” is the name of the module you are creatinng
Control-click (or right-click) on the newly-created group
Choose "New Group" from the context menu
Name the newly-created group Sources
Control-click (or right-click) on the newly-created group
Choose “New File…” from the context menu, and create a new Swift file using the assistant
Select the top-level Xcode project from the Project navigator
In the Editor menu, select “Add Target…”
Select the iOS platform at the top of the assistant, choose the “Static Library” template, and click the Next button
In the “Product Name” field, put the name of the module, matching the name of the “ModuleName” you chose in step 5
Fill in the other fields and click the Finish button
In the Project editor, select the project itself, and then select the Info tab
In the Configurations outline view, expand all build configurations
For each build configuration, in the row for the newly-added target, click “None” and choose “ModuleOverridingBuildSettings” from the pop-up menu
In the Project editor, select the newly-added target, and then select the Build Settings tab
Click “All” in the filter bar to show all build settings
Click on a build setting to select it, and then open the Edit menu and click “Select All”
Press the Delete key on your keyboard to remove all build setting overrides at the target level
Select the Build Phases tab
Expand the “Compile Sources” build phase
Remove the source file that is listed in the “Compile Sources” build phase
From the Project navigator, drag the source file added in step 10 into the “Compile Sources” build phase
In the Project navigator, locate the group named after the target added in steps 12—15 (i.e. “ModuleName” without the “.playgroundbook” extension)
Control-click (or right-click) on this group
Choose “Delete” from the context menu
Select “Move to Trash” in the confirmation dialog to delete the source files

These steps give you a configured, standalone auxiliary or user module. As you’re using the module you’ve just created, it’s important to confirm the following things:

The source files for the auxiliary or user module must be inside of the “Sources” folder in the “ModuleName.playgroundmodule” folder in the “Modules” or “UserModules” folder on-disk. If not, they will not be copied into the final playground book
You must specify explicit target dependencies in the Project editor to ensure that the modules build in the correct order
If you want to use this module from LiveViewTestApp, you must specify an explicit target dependency from LiveViewTestApp to this module

Deleting an Auxiliary or User Module

To delete an auxiliary module or user module, you need to delete three things: the target for the library which builds the module, the group in the Xcode project for the module, and the on-disk folder for the module. This can be accomplished by performing the following steps:

Select the top-level Xcode project from the Project navigator
Select the target for the module you wish to delete
Control-click (or right-click) on the target for the module
Choose “Delete” from the context menu and confirm the deletion
Control-click (or right-click) on the group for the module you wish to delete in the Project navigator (e.g. ModuleName.playgroundmodule)
Choose “Delete” from the context menu
Select “Move to Trash” in the confirmation dialog to delete the source files
Expand the “Supporting Content” group (a child of the

HearSee

Install / Use

README