Search for b-->u transitions in B- --> DK- and B- --> D*K- Decays

We report results from an updated study of the suppressed decays B- to D K- and B- to D* K- followed by D to K+pi-, where D() indicates a D()0 or an anti-D()0 meson, and D decays to D pi0 or D gamma. These decays are sensitive to the CKM unitarity triangle angle gamma, due to interference between the b to c transition B- to D()0K- followed by the doubly Cabibbo-suppressed decay D0 to K+pi-, and the b to u transition B- to anti-D()0K- followed by the Cabibbo-favored decay anti-D0 to K+pi-. We also report an analysis of the decay B- to D()pi- with the D decaying into the doubly Cabibbo-suppressed mode D to K+pi-. Our results are based on 467 million Upsilon (4S) to B anti-B decays collected with the BaBar detector at SLAC. We measure the ratios R() of the suppressed ([K+pi-]D K-/pi-) to favored ([K-pi+]D K-/pi-) branching fractions as well as the CP asymmetries A() of those modes. We see indications of signals for the B- to D K- and B- to DK-, D*->D pi0 suppressed modes, with statistical significances of 2.1 and 2.2 sigma, respectively, and we measure: R{DK} = (1.1\pm 0.6 \pm 0.2)\times 10^{-2}, A{DK} = -0.86 \pm 0.47
^{+0.12}{-0.16}, R*{(D pi0)K} = (1.8\pm 0.9 \pm 0.4)\times 10^{-2}, A*{(D pi0)K} = +0.77 \pm 0.35\pm 0.12, R*{(D gamma)K} = (1.3\pm 1.4\pm 0.8 )\times 10^{-2}, A*{(D gamma)K} = +0.36 \pm 0.94\ ^{+0.25}{-0.41}, where the first uncertainty is statistical and the second is systematic. We use a frequentist approach to obtain the magnitude of the ratio r_B = |A(B- to anti-D0 K-) / A(B- to D0K-)|= (9.5^{+5.1}_{-4.1})%, with r_B<16.7% at 90% confidence level. In the case of B- to D* K- we find r*B = |A(B^- to anti-D0 K-)/A(B- to D0 K-)|=(9.6^{+3.5}{-5.1})%, with r*_B<15.0% at 90% confidence level.