When you complete the second quest Taking Care of Business you will become a member of the Thieves Guild. You don't have to do Vex's or Delvin's radiant quests to get the option, but merely starting the Thieves Guild main quest line does not make you a member of the Thieves Guild. If you're a member of the Thieves Guild then you will have bribery dialogue options with the guards regarding your crime.
