Then I stumbled upon this, which did work:
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
// Set up the action bar.
final ActionBar actionBar = getActionBar();
actionBar.setDisplayOptions(0); // <----- THIS IS THE PART THAT WORKED.
actionBar.setNavigationMode(ActionBar.NAVIGATION_MODE_TABS);
// etc...
}
Also worth noting that in my case I'm not displaying an options menu, via:
@Override
public boolean onCreateOptionsMenu(Menu menu) {
return false;
}