Ken Arroyo Ohori

Open 3D city models for Mexico

Wed, 12 Aug 2026 11:00:00 -0600

Mexico’s national mapping agency INEGI provides topographic and elevation datasets covering the whole country, so I got curious about whether they could be used to create 3D city models using only open data. There’s one catch: the topographic dataset is missing two of the most important feature types for 3D city models, namely individual building footprints and road polygons. My method works around this by deriving building footprints directly from the elevation data (subtracting the terrain model from the surface model and region-growing from the resulting heights) and generating road polygons from the empty spaces between city blocks in the topographic data.

The result is a semantically rich CityJSON model and a textured OBJ model, including buildings, roads, plant cover, water bodies and terrain. In central-western Mexico City, where I tested the method, the building footprints are often more accurate than those from global datasets (Microsoft, Google), particularly for non-rectilinear landmarks such as the Torre Mayor and the Mexico City World Trade Center.

The work is described in more detail in the paper “Creating 3D city models of Mexican cities based on open data”, presented at the Conference on Geoinformation 2025 in Mérida, Mexico. All the code is open source on GitHub.

I’ve also made the output data from the paper freely available. It covers the tile of central-western Mexico City in both formats (the CityJSON model plus the OBJ model and its terrain): e14a39b3.zip.

If you’d like the rest of Mexico City or any other region where high-resolution elevation data is available, or if you’d like to collaborate further on this, get in touch!

#30DayMapChallenge

Tue, 22 Nov 2022 11:47:00 -0600

I contributed 3 maps for my research group’s joint effort in the #30DayMapChallenge. They’re posted in the group’s Twitter, but I’m linking the results from here as well.

Day 10: OpenStreetMap

This map shows the number of buildings footprints available in OpenStreetMap per H3 cell. It’s interesting to see that some regions of the world are much better mapped than others and that the data doesn’t correlate so much with population or wealth.

The script to generate the data is available here.

Day 21: Kontur population dataset

This map shows the world’s biggest cities, as independently computed from the Kontur population dataset. The first step was to classify all H3 cells into urban and non-urban based on a population density threshold (default 5000/km^2). Urban cells are then aggregated into cities as long as they form nearly contiguous regions (up to a maximum cell separation threshold).

Because of the methodology, it is possible to generate the world’s biggest cities using a uniform standard, which contrasts with the inconsistent way in which different countries compute their own statistics. If you dislike urban sprawl as much as me, you’ll be pleased to see that this map shows the true nature of suburban development.

The script to generate the data is available here.

Day 23: Movement

This map, inspired by Chronotrains, shows how far you can travel in Mexico City using public transit in an hour using precomputed isochrones (15-, 30-, 45- and 60-minutes). The isochrones are computed for all metro, BRT, light rail, suburban train and cable car stops using data from the Mexico City static GTFS and a custom method.

The code to generate the map and compute the isochrones is available here.

How to annoy submitters to your scientific conference or journal

Mon, 10 Jul 2017 18:23:00 -0500

It can sometimes seem like conference organisers and publishers are working hard to impose on authors as many pointless rules as possible. So, I decided to compile a list of my favourite ones.

Updated on 11 July 2017

Conferences

Change your submission deadlines many times with no advance notice, preferably after the previous deadline has already passed. Everyone loves scrambling to meet a deadline, then learning that they could have avoided working on their holidays.
Tell submitters to your conference that the best papers will be published in a high-profile journal. Only after the conference you tell them that the papers will have to be extended and go through additional rounds of reviews.

Paper content

Limit the use of excerpts from other people’s work. Fair use be damned, that’s the way to go!
Force a standard list of sections. All work is basically the same, so it all fits the same format. This in no way undermines the ability of authors to explain their work.
Ask authors to spend hours looking for obscure bibliographical details for each of their sources. Everyone needs to know a defunct publisher’s old address.
Journal names should be abbreviated according to a list. This is in no way a ridiculous practice. Every character saved counts!

LaTeX vs. Word

Don’t provide a LaTeX template. E-mailing Microsoft Word files back and forth is the best form of collaboration. Merging the changes of multiple people is always great fun.
If you don’t have technical expertise to deal with LaTeX files, just tell people you accept LaTeX but then ask for a .doc file once a paper is accepted! Sunk costs will ensure that authors will painstakingly do the conversion at no cost to you.
Implement a live-building LaTeX system rather than doing PDF uploads. Spending a day debugging a LaTeX file in 15 minute increments is a great use of researchers’ time.
Provide a LaTeX template that doesn’t match the Word template. Then ask authors to play with their manuscript until it does.

Language and formatting

Require a specific variety of English. British English and American English are mutually unintelligible, so it makes sense for your readers to only understand one of them. Besides, no one is more capable of dealing with this than a non-native English speaking scientist!
Don’t allow subfigures. It’s much better to have authors merge them into a single makeshift figure with awkwardly sized labels and mismatched fonts.
Rasterised text in figures is always great. Forcing users to put it there by limiting what people can put in a caption is just smart thinking.
Force authors to submit their figure files named according to their figure number. Everyone knows their figures by number rather than by their content, right?
Limit colour figures. Who cares if it makes your figures much harder to read. Readers always use the printed versions of a journal or prints a paper on dot-matrix printers, right?
Limit file sizes to a few MB. Low-quality graphics are far preferable over a largish PDF file.
Tell authors to push all their figures to the end of the paper. Going back and forth while reading is best.
Limit image file formats to cutting-edge EPS or TIFF.

Submission, reviews and publishing

Charge a few thousand dollars/euros for open access publishing. Professional editing and hosting are all super expensive. There’s clearly no hint of monopoly pricing there. None whatsoever.
Force submitters to suggest a few friendly reviewers. Surely this in no way undermines the impartiality of a review!
Allow reviewers to force authors to cite the reviewers’ own work. Editors are obviously in no position to stop this bad practice.
Reject papers without a review for no obvious reason. Bonus points if you do so using a generic letter with a list of potential reasons for the authors to guess which one applies to them.
Don’t accept zip files. Instead, make authors upload every figure separately. Just click on add file, browse and upload each time. Piece of cake!
Force authors to anonymise the paper but don’t scrub the author information from the file metadata. It’s just fun to leave it there for technically-inclined reviewers to find!

Methodological problems using Mexico City's pollution data

Tue, 10 Jan 2017 21:08:00 -0600

Pollution in Mexico City has been a problem for decades. I remember my childhood in the early 1990s as a time of always-grey skies, cheap leaded petrol, and almost unbelievable tales of birds falling dead from the sky. In 1992, Mexico City earned the dubious distinction of the most polluted city on the planet. Many inhabitants of the city were outraged¹.

Nowadays the skies are mostly blue and the situation is much improved, which is a real achievement considering the rather inauspicious location of Mexico City in a closed valley more than 2 km above sea level². This improvement was no coincidence. Tough actions followed the worst years of the pollution crisis, including closing down an oil refinery, moving certain factories outside the city and forcing all cars to install catalytic converters and do stringent twice-yearly emission controls.

However, the most forward-looking reaction was perhaps the creation of a city-wide pollution monitoring system in 1986, which is now known as SIMAT. The system is acknowledged as a big success and is widely used. For one, if the pollution levels reach a certain level on a given day, restrictions on car circulation are imposed for the following day. Many people in Mexico City check the pollution levels more frequently than the weather forecast³.

While the raw pollution data is generally regarded as accurate and reliable, it has some quirks:

Pollution monitoring stations have been moved many times and the number of stations has constantly fluctuated.
Not all pollutants are measured at every station.
Stations are regularly taken offline for maintenance.

All of the above means that many (most?) users of Mexico City’s pollution data use it incorrectly including the Mexico City and the Mexican federal governments. This blog post documents a couple of bad examples that show what I think are the main incorrect uses of the data and hopefully helps others to use it better.

Statistically weak statistics

An intergovernmental working group known as the Comisión Ambiental de la Megalópolis (the Environmental Commission for the Megalopolis) issues environmental alerts based on the maximum pollution level in the city. This is statistically suspect: the more stations that are online, the more likely that a local pollution spike will be detected and an environmental alert will be issued. In turn, this creates a perverse incentive for the government to reduce the number of pollution monitoring stations and to locate them away from polluted areas.

For instance, the station in Xalostoc (an industrial area in the Northeast of the metropolitan area) is often the most polluted in the city. If this station is (temporarily) offline, the chances of a pollution alert being declared decrease substantially. I am certain that if more stations were integrated into the system, particularly in other industrial areas such as Cuautitlán Izcalli, Iztapalapa, or the neighbouring Toluca or Mezquital valleys (which are also part of the megalopolis), it would cause the number of pollution alerts to increase.

For another example of statistically weak statistics, consider the pretty mosaics showing the maximum pollutant data for every day during the past years. They also use only the maximum and fail to take into account the general increase in the number of stations throughout the years.

Not dealing correctly with holes in the data

Raw pollutant levels are hard to grasp for non-specialists. Thus, the government of Mexico (like most other governments) has defined a pollution index called IMECA. Its value is calculated independently for every pollutant (CO, CO₂, NO₂, ozone, PM₁₀, PM_2.5, etc.), but the only number that is easily available and widely broadcast to the population is the maximum for all pollutants per station⁴.

When you consider this issue together with the fact that not all stations measure all pollutants, another problem becomes apparent: the IMECA value for each station is only really a lower bound (as opposed to its actual but unknown value) and these values cannot be interpolated. And yet, they are directly interpolated by most services, including the otherwise great Hoyo de Smog and the Netatmo app. This is a completely wrong way to provide pollution data at every location⁵.

As another bad example, consider the maps shown in SIMAT’s website. Apparently, my home borough of Coyoacán (circled in red) is cleaner than the surrounding area, right? In reality, the monitoring stations in the borough do not measure PM₁₀, which is the main pollutant in these cold winter days⁶. This is doubly misleading in the borough/municipality view shown on the right where (again) boroughs/municipalities with fewer stations and measuring fewer pollutants are generally shown as cleaner.

Perhaps I’ll end up creating my own real-time pollution map someday. That would be a nice small project.

Appendix: getting the data

but the cynical nature of chilangos made more than a few wear it with pride… ↩
No other megacity is in a similarly difficult location. ↩
which is perhaps not a surprise considering the predictably mild weather of Mexico City ↩
which is another case of statistically weak statistics… ↩
A still-simple-but-much-better solution: interpolate IMECA values per pollutant, then use the maximum everywhere. ↩
due to thermal inversions trapping the pollution in the valley ↩

Calling C++ from Swift

Mon, 05 Dec 2016 17:44:00 -0600

When I was recently working on azul, a CityGML viewer for Mac, my first thought was to make a pure Swift 3 project. But alas, I soon found out that Apple’s XmlParser is awfully slow. Compared to val3dity, the first working version of the viewer was at least 10x slower in parsing XML.

The solution to this problem was pretty simple: using pugixml as an XML parsing backend instead¹. However, pugixml is written in C++ and so it cannot be called directly from Swift. In my field of GIS research this is actually a very typical problem since most useful libraries are written in C/C++ (e.g. Boost, CGAL, GDAL, GEOS). Considering the relatively short time Swift has been out and its admittedly limited appeal, I suspect this is the case for a lot of people.

So, how do you call C++ code from Swift 3? There are different ways to do it, but in my opinion the best involves an Objective-C++ double wrapper with a pure Objective-C header.

A double wrapper

I will use examples directly taken from azul. So, assuming you have a C++ class CityGMLParser, you simply need to create an Objective-C header CityGMLParserWrapperWrapper.h:

struct CityGMLParserWrapper;

@interface CityGMLParserWrapperWrapper: NSObject {
  struct CityGMLParserWrapper *parserWrapper;
}

- (id) init;
- (void) dealloc;

Note that we use a forward declaration of the first wrapper CityGMLParserWrapper and store a pointer to it in the second wrapper. In this way, we can keep the header as pure Objective-C and thus call it from Swift.

Next, we define its implementation in an Objective-C++ CityGMLParserWrapperWrapper.mm:

#import "CityGMLParserWrapperWrapper.h"
#import "CityGMLParser.hpp"

struct CityGMLParserWrapper {
  CityGMLParser *parser;
};

@implementation CityGMLParserWrapperWrapper

- (id) init {
  if (self = [super init]) {
    parserWrapper = new CityGMLParserWrapper();
    parserWrapper->parser = new CityGMLParser();
  } return self;
}

- (void) dealloc {
  delete parserWrapper->parser;
  delete parserWrapper;
}

Note that CityGMLParser is the C++ class, and that we need to create the two wrapper levels in init and delete them (C++ style) in the dealloc. This file can contain as much C++ as Objective-C++ allows.

Finally, you likely need to add a bridging header that contains the import statement of your wrapper wrapper so that it can be used in Swift. If you use Xcode, this needs to be set in the Build Settings tab of the project settings. It simply needs to contain something like:

#import "CityGMLParserWrapperWrapper.h"

Passing information between C++ and Swift code

Now, you will probably need to pass information between the C++ and the Swift code. This is relatively straightforward but is one of the areas where Swift offers little syntactic sugar.

If you’re using a POD data type, this is easiest. Just add appropriate getters and setters in your wrapper wrapper. These should make calls of the form return parserWrapper->parser->... or parserWrapper->parser->... = ....

So, let’s look directly at something a bit more difficult. In order to read an array or another memory-contiguous sequence of elements of known size.

A couple of arrays

In my C++ code, I have a couple of arrays that keep the bounding box surrounding my data:

float minCoordinates[3];
float maxCoordinates[3];

In order to access these, I have a couple of method declarations in my Objective-C wrapper:

- (float *) minCoordinates;
- (float *) maxCoordinates;

Note that we will return a pointer. This will be the address of the first element of each array. Then, the implementation of these methods is straightforward:

- (float *) minCoordinates {
  return parserWrapper->parser->minCoordinates;
}

- (float *) maxCoordinates {
  return parserWrapper->parser->maxCoordinates;
}

Iterators

Now let’s look at a more complex example. In azul, I needed a way to obtain much more complex data structures that were created in C++. These data structures consisted of std::lists of custom classes containing various attributes.

In such cases where you don’t know exactly where things might be stored², the easiest way to transfer information is by defining custom iterators and to pass the information object by object and attribute by attribute.

So, in my C++ code I created various iterators such as this one:

std::list<CityGMLObject>::const_iterator currentObject;

Then, in the wrapper wrapper I defined appropriate methods to initialise, advance and check if the iterator has ended:

- (void) initialiseObjectIterator;
- (void) advanceObjectIterator;
- (BOOL) objectIteratorEnded;

These can be implemented by simply calling the respective C++ iterator functions:

- (void) initialiseObjectIterator {
  parserWrapper->parser->currentObject = parserWrapper->parser->objects.begin();
}

- (void) advanceObjectIterator {
  ++parserWrapper->parser->currentObject;
}

- (BOOL) objectIteratorEnded {
  if (parserWrapper->parser->currentObject == parserWrapper->parser->objects.end()) {
    return true;
  } return false;
}

Strings

Now, let’s look at how to extract somewhat more complex C++ variables such as std::strings or other types that can be extracted as contiguous blocks of memory that don’t have a fixed length.

In the case of an std::string, the trick is to use its C representation as a const char * and to also receive a place to pass its length to the Swift code. A getter in the wrapper wrapper can then look like this:

- (const char *) currentObjectTypeWithLength: (unsigned long *)length;

Note that we use an unsigned long since that is equivalent to a Swift UInt. The implementation would be like this:

- (const char *) currentObjectTypeWithLength: (unsigned long *)length {
  *length = parserWrapper->parser->currentObject->type.size();
  return parserWrapper->parser->currentObject->type.c_str();
}

Unlike in the other examples, the Swift code is not trivial and so I will explain it as well:

var objectTypeLength: UInt = 0
let firstElementOfObjectTypeBuffer = UnsafeRawPointer(cityGMLParser.currentObjectType(withLength: &objectTypeLength))
let objectTypeData = Data(bytes: firstElementOfObjectTypeBuffer!, count: Int(objectTypeLength)*MemoryLayout<Int8>.size)
objects.last!.type = String(data: objectTypeData, encoding: .utf8)!

There are a few things to see here. We first define the variable where we will store the string length objectTypeLength and pass it as a pointer using &objectTypeLength. The return value is immediately cast to an UnsafeRawPointer.

Then, we use the Swift Data to store the contents of the buffer at the location of the returned pointer. And finally, we can create a Swift String using the buffer data.

which is also used in val3dity ↩
or it’s cumbersome to find that out ↩

Swift 3 and Metal

Thu, 17 Nov 2016 16:04:00 -0600

As a follow-up to the previous post, perhaps it’s also helpful to explain the easiest way to draw to a Cocoa view using Metal rather than OpenGL¹. And as a bonus, I also explain how to select OpenGL or Metal rendering on the fly.

Some background

Apple has never been diligent in keeping up with new OpenGL versions (or the latest graphics cards for that matter), but the situation has reached a pretty bad point. Even the newest hardware supports only six-year-old OpenGL 4.1. By comparison, Metal is under constant development and has tons of new features announced every WWDC. But perhaps for the same reason there’s a dearth of documentation and good, introductory level and up-to-date examples².

Now, to be honest, I’d prefer to use something open and cross-platform like Vulkan, but it’s still a no-show on macOS (and I don’t expect it to change in the future). Or I could switch to Windows. Just kidding. I’d of course switch to Linux instead.

Drawing to an MTKView

By far the easiest way to draw to a Cocoa view is to subclass MTKView, part of MetalKit and available from iOS 9 and El Capitan onwards. This is similar to NSOpenGLView for OpenGL, and it means that most things are managed for you automatically. So, you probably need to set up a class like this:

import Metal
import MetalKit

class MetalView: MTKView {
  var commandQueue: MTLCommandQueue?
  var renderPipelineState: MTLRenderPipelineState?
  var depthStencilState: MTLDepthStencilState?
}

Afterwards, create the init() method and set up the Metal device, MTKView properties, command queue, render pipeline, shaders and depth stencil there. The same goes if you need textures or other stencils.

required init(coder: NSCoder) {
  
  super.init(coder: coder)
  
  // Device
  device = MTLCreateSystemDefaultDevice()
  
  // View
  clearColor = MTLClearColorMake(1.0, 1.0, 1.0, 1)
  colorPixelFormat = .bgra8Unorm
  depthStencilPixelFormat = .depth32Float
  
  // Command queue
  commandQueue = device!.makeCommandQueue()
  
  // Render pipeline
  let library = device!.newDefaultLibrary()!
  let vertexFunction = library.makeFunction(name: "vertexTransform")
  let fragmentFunction = library.makeFunction(name: "fragmentLit")
  let renderPipelineDescriptor = MTLRenderPipelineDescriptor()
  renderPipelineDescriptor.vertexFunction = vertexFunction
  renderPipelineDescriptor.fragmentFunction = fragmentFunction
  renderPipelineDescriptor.colorAttachments[0].pixelFormat = colorPixelFormat
  renderPipelineDescriptor.depthAttachmentPixelFormat = depthStencilPixelFormat
  do {
    renderPipelineState = try device!.makeRenderPipelineState(descriptor: renderPipelineDescriptor)
  } catch {
    Swift.print("Unable to compile render pipeline state")
    return
  }
  
  // Depth stencil
  let depthSencilDescriptor = MTLDepthStencilDescriptor()
  depthSencilDescriptor.depthCompareFunction = .less
  depthSencilDescriptor.isDepthWriteEnabled = true
  depthStencilState = device!.makeDepthStencilState(descriptor: depthSencilDescriptor)
}

Finally, create the draw method, which will be called either automatically every frame, automatically with a view notification or manually, depending on how you set up isPaused and enableSetNeedsDisplay.

override func draw(_ dirtyRect: NSRect) {
    let commandBuffer = commandQueue!.makeCommandBuffer()
    let renderPassDescriptor = currentRenderPassDescriptor!
    let renderEncoder = commandBuffer.makeRenderCommandEncoder(descriptor: renderPassDescriptor)
    
    renderEncoder.setFrontFacing(.counterClockwise)
    renderEncoder.setDepthStencilState(depthStencilState)
    renderEncoder.setRenderPipelineState(renderPipelineState!)
    
    // Draw something...
    
    renderEncoder.endEncoding()
    let drawable = currentDrawable!
    commandBuffer.present(drawable)
    commandBuffer.commit()
  }

Selecting Metal or OpenGL rendering on the fly

If you’re excited about the performance of Metal (like me) but want to support Mac hardware older than 2012 (like me), you probably want to select between custom subclasses of MTKView and NSOpenGLView on the fly. In that way, you can probably throw away the OpenGL code in a few years without major code rewrites.

For me, the first step was to isolate all OpenGL- and Metal-specific code into these two subclasses. After that, create a simple placeholder NSView in Interface Builder and kept a reference to it (here view). Now, when your application loads you can do something like this:

func applicationDidFinishLaunching(_ aNotification: Notification) {
  
  let processInfo = ProcessInfo.processInfo
  let sierraOrHigher = processInfo.isOperatingSystemAtLeast(OperatingSystemVersion(majorVersion: 10, minorVersion: 12, patchVersion: 0))
  if sierraOrHigher, let defaultDevice = MTLCreateSystemDefaultDevice() {
    let metalView = MetalView(frame: splitView.subviews[1].frame, device: defaultDevice)
    metalView.controller = self
    splitView.removeArrangedSubview(splitView.arrangedSubviews[1])
    splitView.insertArrangedSubview(metalView, at: 1)
    view = metalView
    window.makeFirstResponder(metalView)
  } else {
    let attributes: [NSOpenGLPixelFormatAttribute] = [
      UInt32(NSOpenGLPFAAccelerated),
      UInt32(NSOpenGLPFAColorSize), UInt32(24),
      UInt32(NSOpenGLPFADoubleBuffer),
      UInt32(NSOpenGLPFADepthSize), UInt32(32),
      UInt32(0)
    ]
    let pixelFormat = NSOpenGLPixelFormat(attributes: attributes)
    let openGLView = OpenGLView(frame: splitView.subviews[1].frame, pixelFormat: pixelFormat)
    openGLView!.controller = self
    splitView.removeArrangedSubview(splitView.arrangedSubviews[1])
    splitView.insertArrangedSubview(openGLView!, at: 1)
    view = openGLView
    window.makeFirstResponder(openGLView)
  }
}

Note a few things. If you want to support the newest Metal features as of 2016³, you want to test for Sierra support. My application crashes on 10.11 even on Metal-compatible hardware⁴, so I’ve forced OpenGL on 10.11 or lower. The easiest way to test for Metal support is to check the return of MTLCreateSystemDefaultDevice().

Also, substituting a view on the fly is tricky business. Substituting only the view will not have any effect, so you need to remove it from its parent view (in this case splitView, an instance of NSSplitView) and re-add your subclass of MTKView or NSOpenGLView.

Finally, for this to work you need to use different initialisers in your MTKView and NSOpenGLView subclasses. These should receive the view’s frame and other variables rather than an instance of NSCoder.

// In the MTKView subclass:
override init(frame frameRect: CGRect, device: MTLDevice?) {
  super.init(frame: frameRect, device: device)
  // Same as before...
}

// In the NSOpenGLView subclass:
override init?(frame: NSRect, pixelFormat: NSOpenGLPixelFormat?) {
  super.init(frame: frame, pixelFormat: pixelFormat)
  // Same as before...
}

For my application, this resulted in a roughly 10x performance improvement over OpenGL. Your mileage may vary. ↩
The clear exception to this is Apple’s Adopting Metal 2016 WWDC sessions. See part 1 and part 2. Don’t forget to check out the sample code. ↩
Why is Apple not putting clear version numbers?! ↩
As does Apple’s own Adopting Metal Sample code… ↩

Swift 3 and OpenGL

Sat, 05 Nov 2016 15:17:00 -0600

Learning Swift has long been in my to-do list. So, when I recently undertook the project to write a CityGML viewer for Mac (to be released soon), I decided the time to learn Swift was now. Fortuitously, Apple had just released Swift 3, and since I like shiny new things, I decided to learn Swift 3 directly.

However, I soon realised that Apple’s documentation is very spotty. Despite the new look of the documentation pages, almost all sample code and examples still use Objective-C. Moreover, interacting with low-level C APIs like OpenGL is more than a little complex, requiring several conversions and the use of pointer-like data types.

Thus, I hope to document a few of the issues that stumped me and how I solved them. In this post, I will explain how to draw OpenGL to an NSOpenGLView or to a custom NSView. For more info, see Apple’s documentation (which uses Objective-C). Heads up to this Stack Overflow question and answers for how this works with Swift 2.

Using NSOpenGLView

The easiest way to draw OpenGL is to subclass NSOpenGLView. In this way, most things are managed for you automatically. So, you probably need to set up a class like this:

import Cocoa
import OpenGL
import GLKit

class OpenGLView: NSOpenGLView {
  var displayLink: CVDisplayLink?
}

Afterwards, create the init?() method and set up the OpenGL pixel format, context and swap interval there.

required init?(coder: NSCoder) {
  super.init(coder: coder)
  
  let attributes: [NSOpenGLPixelFormatAttribute] = [
    UInt32(NSOpenGLPFAAccelerated),
    UInt32(NSOpenGLPFAColorSize), UInt32(32),
    UInt32(NSOpenGLPFADoubleBuffer),
    UInt32(NSOpenGLPFAOpenGLProfile),
    UInt32(NSOpenGLProfileVersion3_2Core),
    UInt32(0)
  ]
  self.pixelFormat = NSOpenGLPixelFormat(attributes: attributes)
  self.openGLContext = NSOpenGLContext(format: pixelFormat, share: nil)
  self.openGLContext?.setValues([1], for: NSOpenGLCPSwapInterval)
  
}

Then, create the prepareOpenGL() method and its callback. This will be called before the first render and is meant to initialise the state of OpenGL. In this example, we set up CVDisplayLink so that a callback function is called to render every new frame.

Note the pointer types UnsafePointer, UnsafeMutablePointer and UnsafeMutableRawPointer, and how unsafeBitCast is used in order to convert the UnsafeMutableRawPointer to an instance of the current class. This is necessary in order to call any methods.

override func prepareOpenGL() {
    
  func displayLinkOutputCallback(displayLink: CVDisplayLink, _ now: UnsafePointer<CVTimeStamp>, _ outputTime: UnsafePointer<CVTimeStamp>, _ flagsIn: CVOptionFlags, _ flagsOut: UnsafeMutablePointer<CVOptionFlags>, _ displayLinkContext: UnsafeMutableRawPointer?) -> CVReturn {
      unsafeBitCast(displayLinkContext, to: OpenGLView.self).renderFrame()
      return kCVReturnSuccess
    }

  CVDisplayLinkCreateWithActiveCGDisplays(&displayLink)
  CVDisplayLinkSetOutputCallback(displayLink!, displayLinkOutputCallback, UnsafeMutableRawPointer(Unmanaged.passUnretained(self).toOpaque()))
	CVDisplayLinkStart(displayLink!)
}

The renderFrame() method is called by the callback function in the CVDisplayLink. There, you can use any OpenGL drawing functions.

func renderFrame() {
  CGLLockContext(self.openGLContext.cglContextObj!)
  
  // Draw something...

  CGLFlushDrawable(self.openGLContext.cglContextObj!)
  CGLUnlockContext(self.openGLContext.cglContextObj!)
}

The draw method should call renderFrame() as well.

override func draw(_ dirtyRect: NSRect) {
  super.draw(dirtyRect)
  renderFrame()
}

Finally, the deinit() method should stop the display link.

deinit {
  CVDisplayLinkStop(displayLink!)
}

Drawing to a custom NSView

Sometimes, NSOpenGLView is not good enough. For instance, NSOpenGLView doesn’t support context sharing, which is a problem if you want to create full screen applications. In those cases, you need to subclass NSView instead. Apple provides this piece of sample code for that purpose, but it is also written in Objective-C and way out of date.

Subclassing NSView is similar to the previous case. However, you need to keep your own NSOpenGLPixelFormat and NSOpenGLContext.

import Cocoa
import OpenGL
import GLKit

class OpenGLView: NSView {
  var pixelFormat: NSOpenGLPixelFormat?
  var openGLContext: NSOpenGLContext?
  
  var displayLink: CVDisplayLink?
}

The init? method can then be pretty much as before, but it is necessary to make it call you own method to initialise OpenGL (setupDisplayLink()) and the display link. If you want context sharing, you probably want to create another init? method that receives the context to share. Make sure to call makeCurrentContext() on the OpenGL context.

required convenience init?(coder: NSCoder) {
  self.init(coder: coder, context: nil)
}

init?(coder: NSCoder, context: NSOpenGLContext?) {
  // Set up pixel format and context...
  
  super.init(coder: coder)
  openGLContext!.makeCurrentContext()
  setupDisplayLink()
  
  // ...

  NotificationCenter.default.addObserver(self, selector: #selector(reshape), name: NSNotification.Name.NSViewGlobalFrameDidChange, object: self)
}

The overriden methods to lock the focus and to draw the view should set the view of the context to itself.

override func lockFocus() {
  super.lockFocus()
  if openGLContext!.view != self {
    openGLContext!.view = self
  }
}

override func draw(_ dirtyRect: NSRect) {
  if openGLContext!.view != self {
    openGLContext!.view = self
  }
  
  // Actually draw something...
  if !CVDisplayLinkIsRunning(displayLink!) {
    drawView()
  }
}

Finally, the method to draw things (as called by draw() or by the display link callback) should do as before, but also make sure to set the current OpenGL context.

func drawView() {
  CGLLockContext(openGLContext!.cglContextObj!)
  openGLContext!.makeCurrentContext()
  
  // Draw things...

  openGLContext!.flushBuffer()
  CGLUnlockContext(openGLContext!.cglContextObj!)
}

The awful public transport layer of Google Maps

Wed, 27 Jul 2016 00:16:00 -0500

Two days ago, Google released a new version of Google Maps. Most people seem to agree that it’s a nice improvement on an already great service. When I saw it, I immediately loved the cleaner look and the improved readability.

That is, until I activated the public transport layer and saw that it remains as ugly and badly designed as ever.

As someone who is enthusiastic about both maps and public transport, this is disappointing. It’s been more than 10 years since Google Transit first appeared on Google Labs and almost 9 years since it was integrated into Google Maps. And so, I decided to write a ‘name and shame’ post to see how long it takes Google to get their public transport maps up to par.

Among other issues, the Google Maps transit layer suffers from discontinuous, crisscrossing and jerky lines, too much an emphasis on roads, badly placed and coloured labels that are often unreadable, repeated or missing, a lack of line and station names on most zoom levels, and a complete lack of the clean look of the standard Google Maps layer. On Map Maker countries the information is also awfully inconsistent, with duplicated stations, wrong categories, misspelled names, different capitalisation styles and a lot of extraneous information (e.g. ‘Metro xxx’ to refer to a station named xxx).

But don’t take my word for it. Instead have a look at the screen caps below, which compare the pretty good public transport layer in Apple Maps¹ (above) and the awful one in Google Maps (below) in various cities and at different scales. I’m sure you’ll see a multitude of problems too.

Berlin

London

Mexico City

Montreal

New York City

Rio de Janeiro

San Francisco

Sydney

Admittedly, this is one of the few areas where Apple Maps is significantly ahead of Google Maps. Apple Maps’ coverage is rather spotty though. ↩

Dimension-independent programming using C++ templates

Sun, 10 Jul 2016 14:39:00 -0500

It’s one of those typical programming conundrums. Popular (programmer) wisdom dictates that your code can be either efficient but inflexible, or flexible but resource intensive. Trying to do both most often than not fails spectacularly.

However, once in a while, one finds a pretty amazing exception, which on first look can seem almost like magic. That is, until you learn how it works on a technical level. For instance, several years ago I was dumbfounded at how DTrace—a nice tracing framework—manages to achieve zero overhead when it is not enabled.

Some time later, I had a similar feeling about how CGAL Combinatorial maps manage to be both fast and completely dimension independent. The answer lies in clever template meta-programming using recursive C++11 templates, a solution that I then incorporated extensively into my PhD work.

Template meta-programming is a technique that uses templates to generate certain data structures or perform certain computations during the compilation of a program rather than during its execution. This technique is normally used as a way to support generic programming by defining functions that can deal with different data types indistinctly. For instance, I often use this nifty piece of code to easily parse numbers into different data types.

template <typename T>
T string_to_number (const std::string &text, T def_value) {
  std::stringstream ss;
  for (std::string::const_iterator i = text.begin(); i != text.end(); ++i)
    if (isdigit(*i) || *i=='e' || *i=='-' || *i=='+' || *i=='.') ss << *i;
  T result;
  return ss >> result ? result : def_value;
}

However, apart from this common use, templates can also be used to create complex type- and dimension-independent data structures and algorithms. CGAL does this using the Traits programming technique, which exploits C++’s typedef declarations to create custom types. As an example, I used this type of approach in my thesis to convert n-dimensional datasets into (n+1)-dimensional ones. The types of the n- and (n+1)-dimensional datasets are automatically derived based on n, which is passed as a template parameter at compilation time. Since higher-dimensional programming is usually heavily constrained by CPU time during execution, this effectively brings the best of both worlds: dimension-independence and speed.

template <unsigned int dimension>
class Linear_cell_complex_extruder_with_range {
public:
  typedef typename Linear_cell_complex<dimension>::type Lower_dimensional_cell_complex;
  typedef typename Linear_cell_complex<dimension+1>::type Higher_dimensional_cell_complex;
  typedef Linear_cell_complex_extruder_with_range<dimension> Self;
  
private:
  Lower_dimensional_cell_complex ldcc;
  Higher_dimensional_cell_complex hdcc;
};

However, to truly exploit this type of mechanism in dimension-independent programming, you need recursive templates. With these, you can build a complex data structure or solve a problem by iterating dimension by dimension at compilation time.

Let’s consider a similar example to one used in my thesis, a data structure where we want to store a value (represented here by a float, but it can be an arbitrarily complex data structure of its own) for every object (represented here by an Object * pointer) of every dimension. Essentially, this would be a std::map<Object *, float>. First, we define a custom struct named Value_map_of_dimension, which stores the values map for a single (templated) dimension. Note the use of the typedef to define the type before using it.

template <unsigned int dimension>
struct Values_map_of_dimension {
public:
  typedef std::map<Object *, float> type;
  type values_map;
};

Then, we use a templated struct that defines Values_map_tuple_per_dimension_up_to, which will store a dimension-dependent tuple of the previous data structures where each element of the tuple will contain one instance of Value_map_of_dimension. Here, the first template parameter is the dimension and the second is the tuple, which is initialised to an empty one.

template <unsigned int dimension, class Result = std::tuple<>>
struct Values_map_tuple_per_dimension_up_to;

This struct is specialised using a recursive template that prepends the tuple with an instance of Values_map_of_dimension for the current dimension and calls itself with dimension-1.

template <unsigned int dimension, class ... Result>
struct Values_map_tuple_per_dimension_up_to<dimension, std::tuple<Result ...>> {
  typedef typename Values_map_tuple_per_dimension_up_to<dimension - 1, 
    std::tuple<Values_map_of_dimension<dimension>, Result ...>>::type type;
};

Finally, this struct is further specialised with a terminating condition for dimension 0, where the tuple is returned as is.

template <class ... Result>
struct Values_map_tuple_per_dimension_up_to<0, std::tuple<Result ...>> {
  typedef std::tuple<Values_map_of_dimension<0>, Result ...> type;
};

For the compiler to build this data structure correctly, it is crucial to ensure that the last struct is a more complete specialisation of Values_map_tuple_per_dimension_up_to. It should therefore have one fewer template parameter than the previous one.

Using a C++ map or set with geometric data

Sun, 19 Jun 2016 18:25:00 -0500

One of the most typical building blocks of efficient geometric computations is a spatial index. Using a spatial index, it is possible to perform many simple operations quickly, like obtaining a set of unique points. Such an operation can then be used for things like building a topological data structure or indexing more complex objects.

However, properly implementing a spatial index is messy and, frankly, overkill for many practical applications. And yet, computations without one can be mind-bogglingly slow. So, I often resort to a quick hack—using a C++ standard library set or map with a custom comparison function. A set or map offers logarithmic time access to its elements, much like most spatial indices, and it works out of the box for plain old data types like float or int. Moreover, in order for you to use it with any class you’re working with, it only needs a custom comparison functor¹ that defines an ordering of its elements².

So, let’s look at an example of some code I’m currently working on. A CGAL Point_d is a simple data structure for a d-dimensional point. I personally want to use these kinds of points to quickly access unique geometric objects embedded at those locations, and so I created a custom comparison functor for them. It orders points first by their dimension and then lexicographically by their coordinates:

struct Point_d_comparator {
  bool operator()(const Point_d &p1, const Point_d &p2) const {
    
    if (p1.dimension() < p2.dimension()) return true;
    if (p1.dimension() > p2.dimension()) return false;
    
    Point_d::Cartesian_const_iterator p1_coordinate = p1.cartesian_begin()
    Point_d::Cartesian_const_iterator p2_coordinate = p2.cartesian_begin();
    while (p1_coordinate != p1.cartesian_end()) {
      if (*p1_coordinate < *p2_coordinate) return true;
      if (*p1_coordinate > *p2_coordinate) return false;
      ++p1_coordinate;
      ++p2_coordinate;
    } return false;
  }
};

In a simple example, such a comparison functor can then be used to define a set of unique points (as defined by their coordinates). For instance, this allows you to assign unique IDs to a location.

std::set<Point_d, Point_d_comparator> vertices;

In a more complex example, the same comparison functor can be used to index a set of edges (as defined by a unique start-end pair of points). I personally use this kind of structure to link a set of adjacent polygons by their common edges and to check if such a set forms a quasi-manifold in 3D. Note that you should make sure to insert the two end-points in a well-defined order, such as the lexicographically smallest first!

std::map<Point_d, std::map<Point_d, int, Point_d_comparator>, Point_d_comparator> edges;

For a future blog post, how to do something similar with a more efficient (constant time) C++11 unordered_map and a custom hashing function.

A functor is an object that behaves like a function. In C++, a functor is a class or struct that does this by overloading the () operator. Unlike function pointers, they are fast since they are generated at compilation time. ↩
Most people choose to mimic the value of the < (less than) operator, which is what set and map do for plain old types. ↩