Both.  In some places the sketchy line layer was enough to select what I needed for a base layer, but in other places I preferred to just color normally, like the eyes.