Bacterial gene expression regulation occurs mostly during transcription, which has two main rate-limiting steps: the close complex formation, when the RNA polymerase binds to an active promoter, and the subsequent open complex formation, after which it follows elongation. Tuning these steps kinetics by the action of, e.g., transcription factors, allows for a wide diversity of dynamics. For example, adding autoregulation generates single-gene circuits able to perform more complex tasks. Using stochastic models of transcription kinetics with empirically validated parameter values, we investigate how autoregulation and the multi-step transcription initiation kinetics of single-gene autoregulated circuits can be combined to fine tune steady state mean and cell-to-cell variability in protein expression levels, as well as response times. Next, we investigate how they can be jointly tuned to control complex behaviors, namely, time counting, switching dynamics, and memory storage. Overall, our finding suggests that, in bacteria, jointly regulating a single-gene circuit’s topology and the transcription initiation multi-step dynamics allows enhancing complex task performance.