Welcome to an exploration of Metal GPU programming for iOS and macOS. Introduced at WWDC 2014, Metal is a powerful, high performance, low overhead graphics API for both iOS and macOS. We'll build Metal Apps ranging from a simple triangle rasterizer to a hybrid raytracer.
Metal resides in the family of low-level graphics APIs alongside Vulkan and Microsoft's DirectX 12. Though Metal is low-level it is largely influenced by the DirectX 11 API. Many of Metal's rasterization state structures mimic DirectX 11's, even considering Apple favored OpenGL until the late 2000's. Nowadays, OpenGL is no longer considered a relevant API on Apple platforms. At WWDC 2018 Apple announced the deprecation of OpenGL, crowning Metal as the de-facto graphics API on Apple platforms.
Metal's native Objective-C interface is used throughout the examples. The source is C++11 with a small amount of Objective-C, all inside a single Objective-C++ file. No headers or extra libraries are necessary to build the examples, and on macOS you can build and execute directly from the terminal. To manage certificates and device builds we maintain an Xcode project for iOS.
Full source for each tutorial is on gitlab and linked at the end of each article.
Clean Slate
Let's begin with the simplest Metal app: a screen clear. Starting with a simple app allows us to cover UI boilerplate in one tutorial and stands on its own as a clean slate for experimentation.
As of Xcode 9 iOS Simulator 11.0, simulated Metal is not supported and requires a physical device. Since this series covers both macOS and iOS, the examples will still run on a Mac. This is fortunate as getting applications onto iOS devices requires an Apple developer account or rooted device.
This series is a bit different from others. We use Objective-C++, focus on both macOS and iOS, use the lowest-level tools available, and forgo Storyboards and XIBs. On the last point, there's little information on programmatically constructing interfaces with UIKit and Cocoa using Objective-C++. For many apps a native UI isn't required. For example, we rarely need storyboards when building scientific visualization or gaming applications. Storyboards add extra overhead and make learning Metal more difficult. We build the interface and all supporting UI programmatically.
Automatic reference counting (ARC) is not used. Memory management is handled explicitly and can be debugged through instruments. Some links for brushing up on Objective-C memory management are given in the references.
Familiarity with C++ and Objective-C is assumed. UIKit and Cocoa code description is minimal as we focus solely on Metal.
Project Set Up
macOS
The macOS build is an 8 line makefile:
TARGET = ex
CXXFLAGS += -O0 -g -std=c++1z
LFLAGS += -framework Cocoa -framework Metal -framework MetalKit\
-framework QuartzCore
$(TARGET): main.mm
$(CXX) $(CXXFLAGS) main.mm $(LFLAGS) -o $(TARGET)
clean:
-$(RM) -rf $(TARGET).dSYM/ $(TARGET)
To build, simply type make into a terminal at the root of the project.
This Makefile is used for all tutorials. Note the lack of CPU optimization (-O0) and addition of debug symbols (-g). We prefer no optimization and symbols since we are focussed on the GPU and want to debug the CPU easily using lldb. That said, all tutorials work with CPU optimization (-O2).
Also note that we have disabled errors inside the clean target. We don't want make clean to generate errors if clean targets are not present. We could make the recipe line silent though that has no bearing on error output.
For a well-rounded introduction to Makefiles, I recommend Stallman and McGrath's GNU Make A Program for Directed Compilation.
iOS
For iOS we use Xcode projects. Xcode allows easy interaction with devices and certificates. Here's a rundown of setting up a minimal iOS Xcode project.
Creating an iOS Project from Scratch
Start Xcode and you should see the standard Welcome to Xcode screen.
- Select 'Create a new Xcode Project' button
- Select 'Game' project.
- Select Language: 'Objective-C', and Technology: 'Metal'
- Finish the wizard.
Remove all the source files that created (*.h, *.m, *.metal) and delete the storyboard files (*.storyboard). Keep Assets.xcassets as that is where we store launch images.
Delete 'Launch screen interface file base name' and 'Main storyboard file base name' from info.plist. Neither apply for programmatically generated UI.
Check 'Requires full screen' under Project Properties, General, Deployment Info. Project properties are found by clicking on your project in the 'Navigator' on the left-hand side of Xcode.
In the same properties screen under General, App Icons and Launch Images, populate Launch Images Source. Use the Assets directory associated with Assets.xcassets. If you wish, use LaunchImage Generator to generate placeholder screens. If Launch Images Source is not set you will notice visible bars at the top and bottom of your device's screen (pre-iOS 5). As of this writing the linked LaunchImage generator doesn't generate iphone X images. To do so, create 1125x2436 (portrait) and 2436x1125 (landscape) images and add them as iPhone X LaunchImages.
In Project Settings under Build Phases, Link Binary with Libraries, link with the frameworks: UIKit.framework, Metal.framework, and QuartzCore.framework. Quartz is required for CAMetalLayer and CADisplayLink.
Finish by adding your source files.
Metal Fundamentals
Let's dive into the source. Both iOS and macOS share the same source modulo a few preprocessor directives (main.mm).
Metal Device and Queue Creation
The first step in any Metal app is creating a Metal device. Metal requires device creation on a specific GPU so we use MTLCreateSystemDefaultDevice to target the default GPU. Care should be taken when utilizing multiple GPUs. For example, use MtlCopyAllDevices if you are using an eGPU with VR or plan on using both Discrete and Integrated GPUs in a MacBook.
The following code from our sample creates the device and a command queue:
// Global variable declaration.
id<MTLDevice> g_mtlDevice;
id<MTLCommandQueue> g_mtlCommandQueue;
// ... implementation ...
int renderInit()
{
g_mtlDevice = MTLCreateSystemDefaultDevice();
if (!g_mtlDevice)
{
fprintf(stderr, "System does not support metal.\n");
return EXIT_FAILURE;
}
g_mtlCommandQueue = [g_mtlDevice newCommandQueue];
return EXIT_SUCCESS;
}
We assign a new default device to g_mtlDevice. Since we aren't building a multithreaded renderer we also create a command queue and assign it to g_mtlCommandQueue. In future entries we'll significantly expand this renderInit function.
Let's be sure to clean up:
void renderDestroy()
{
[g_mtlCommandQueue release];
g_mtlCommandQueue = nil;
[g_mtlDevice release];
g_mtlDevice = nil;
}
We call release in reverse order of creation. renderDestroy is called from a macOS/iOS specific dealloc method.
Frame Callback
Consistent frame timing is required for smooth scene animation. iOS' CADisplayLink and macOS' CVDisplayLink were built to synchronize rendering to the refresh rate of the display. Refresh rate synchronization is important and we'll come back to it in a different series. To synchronize with the display on Apple devices, we add DisplayLink callbacks to the main runloop.
// iOS display link callback.
- (void)displayLinkDidFire:(CADisplayLink *)displayLink
{
doRender();
}
// ...
// macOS display link callback
static CVReturn displayLinkCallback(
CVDisplayLinkRef displayLink,
const CVTimeStamp* now,
const CVTimeStamp* outputTime,
CVOptionFlags flagsIn,
CVOptionFlags* flagsOut,
void* displayLinkContext)
{
doRender();
return kCVReturnSuccess;
}
Details for registering these callbacks are covered in the source. For iOS search for addToRunLoop and macOS CVDisplayLinkSetOutputCallback.
Rendering
With a periodic frame callback in hand, lets get some content on the screen. When the display link callback is issued we call the following doRender function:
void doRender()
{
if (!g_nsView.metalLayer)
{
fprintf(stderr, "Warning: No metal layer, skipping render.\n");
return;
}
id<CAMetalDrawable> drawable = [g_nsView.metalLayer nextDrawable];
id<MTLTexture> texture = drawable.texture;
// Assumes consistent 60Hz refresh rate. Not a great assumption.
// We will use mach_absolute_time for animation is later examples.
static float timeSeconds = 0.0;
timeSeconds += 0.0166;
MTLRenderPassDescriptor* passDescriptor =
[MTLRenderPassDescriptor renderPassDescriptor];
passDescriptor.colorAttachments[0].texture = texture;
passDescriptor.colorAttachments[0].loadAction = MTLLoadActionClear;
passDescriptor.colorAttachments[0].storeAction = MTLStoreActionStore;
passDescriptor.colorAttachments[0].clearColor =
MTLClearColorMake(fmod(timeSeconds * 20.0,1.0), 0.3f, 0.3f, 1.0f);
id<MTLCommandBuffer> commandBuffer = [g_mtlCommandQueue commandBuffer];
id<MTLRenderCommandEncoder> commandEncoder =
[commandBuffer renderCommandEncoderWithDescriptor:passDescriptor];
[commandEncoder endEncoding];
[commandBuffer presentDrawable:drawable];
[commandBuffer commit];
}
The first order of business is to obtain our backbuffer. We retrieve it from CAMetalLayer by calling nextDrawable and fail fast if CAMetalLayer has not been created. From the CAMetalDrawable returned from nextDrawable we retrieve a Metal render target by accessing the texture property.
Now we create a MTLRenderPassDescriptor to attach the render target to a rasterization command encoder. While we only specify one color attachment, any reasonable number of color, depth, or stencil attachments are usable. More on this later.
We populate colorAttachment zero with the backbuffer texture that we obtained from CAMetalLayer. Alongside setting the loadAction to MTLLoadActionClear, this tells Metal which texture to clear on load. Normally this color attachment would be written from a shader, but a fixed-function clear is directly supported by Metal. Shaders are introduced in the next tutorial.
The clear color is set using the clearColor property. We animate the clear color in order to tell that the scene is properly rendered beyond the first frame.
Almost finished. Lets create a MTLCommandBuffer to embed our render pass inside. Create one by calling commandBuffer on the global metal queue g_mtlCommandQueue that we created in renderInit. Then call the command buffer's renderCommandEncoderWithDescriptor which begins encoding our render pass. Since we are only using a fixed-function clear, there's nothing to encode in the pass and we immediately call endEncoding.
The last two lines schedule the drawable's present and commit the command buffer to the GPU. Whew! All done.
iPhone Development Hint
iPhone device logs where instrumental in debugging OS-level UIKit issues, so they deserve special mention. To access the iPhone device logs go to Window -> Devices and Simulators in Xcode. The logs have saved me more than once when the App crashes in system libraries.
Conclusion
This wraps up the bulk of the boilerplate required to get iOS and macOS rendering. Most of the UIKit and Cocoa details are found in the source behind TARGET_OS_IPHONE and TARGET_OS_MAC preprocessor definitions, respectively.
Prior to the announcement of Metal most developers assumed Vulkan was going to be the low-level cross-platform API. In hindsight, Metal appears to have been the right step for the industry. Metal strips away much of the verbosity of other APIs and maintains much of the power.
Blog Entry
The content of this article is exclusive to Steem during the monetization period. Afterwards, it's posted on my blog.
Questions, comments and feedback will remain exclusively on Steem. The blog post will permanently link to this post to drive interaction to Steem.
Repository
Gitlab Repository containing the complete source for this example.
I love learning new programming languages and techniques. Even though I am not on a mac, I will need to check this out later.
Keep up the good work and original content, everyone appreciates it!
Your post had been curated by the @buildawhale team and mentioned here:
https://steemit.com/curation/@buildawhale/buildawhale-curation-digest-07-13-18
Congratulations @iauns! You have completed the following achievement on Steemit and have been rewarded with new badge(s) :
Click on the badge to view your Board of Honor.
If you no longer want to receive notifications, reply to this comment with the word
STOPDo not miss the last post from @steemitboard:
SteemitBoard World Cup Contest - France vs Belgium
Participate in the SteemitBoard World Cup Contest!
Collect World Cup badges and win free SBD
Support the Gold Sponsors of the contest: @good-karma and @lukestokes
Congratulations @iauns! You have completed the following achievement on Steemit and have been rewarded with new badge(s) :
Click on the badge to view your Board of Honor.
If you no longer want to receive notifications, reply to this comment with the word
STOP