Abstract: Aiming at the problem that unreasonable use of margin caused by Large Margin Estimation (LME) only chooses the minimum margin in the support set to adjust, we present a modified form of the LME objective function which gives improved results for discriminative training. The modification consists of boosting the likelihoods of paths in the competing hypothesis that have a higher phone error relative to the correct transcript. So that, the minimum margin of all training data becomes smaller. What’s more, we present a new approximate optimization method. When auxiliary function's and objective function's gradient direction phase at the same time, in the close neighborhood, the optimal solution found for the approximate auxiliary function is expected to improve the original objective function as well. The recognition results on the Microsoft corpora can prove our idea.