P450 complex formation and the unusual pharmacokinetics of methylenedioxyphenyl HIV protease inhibitors were examined by in vitro studies using human and rat liver microsomes and by in vivo oral dosing studies. In vitro spectral studies indicated that the formation of a P450 complex having absorbance maxima at 425 and 456 nm was time and concentration dependent; 27-60% of the total P450 was complexed in dexamethasone-induced rat liver microsomes after a 30-min incubation with 100 microM HIV protease inhibitors. Methoxy substitution on the phenyl ring of the methylenedioxyphenyl moiety increased formation of the P450 complex, whereas chlorine substitution markedly decreased the P450 complexation. Kinetic studies on the P450 complex formation indicated that both methoxy and chlorine substitution affected the maximum complex formation rate (Vmax), while it had little effect on Km values (approximately 10 microM). This complexation in human liver microsomes was inhibited markedly by an anti-CYP3A1 antibody. Furthermore, the P450 complex formation resulted in a time-dependent loss of CYP3A-catalyzed marker activities (testosterone 2beta/6beta-hydroxylase) in both rat and human liver microsomes. Collectively, these results point to the involvement of CYP3A isoforms in P450 complexation by methylenedioxyphenyl HIV protease inhibitors. Additionally, after oral administration to rats, one of these HIV protease inhibitors (Compound I), which complexed P450 to the greatest extent, showed no elimination over a period of 500 min after administration of the highest dose. It is suggested that formation of a quasi-irreversible metabolite-CYP3A complex with methylenedioxyphenyl HIV protease inhibitors was responsible for the CYP3A-selective time-dependent loss of catalytic function and the unusual dose-dependent pharmacokinetics after oral administration.