The effect of competitive research funding mechanisms on research productivity in various disciplines was clarified. Previous studies used multivariate analyses, such as multiple regression analysis, and we built on the existing research by controlling for confounding effects using a balancing score. Typically, a propensity score is used for balancing scores, but this approach has received criticism about its external validity. Therefore, the matching sample in this study was visualized using a propensity score, which was compared with the population and the sampling data for the distribution of disciplines. The main findings of this study were as follow: first, it became clear that the matching sample analyzed using a propensity score strikingly differed from the population. Second, this problem was solved using a covariate balancing propensity score and sampling weights, which estimated the population’s average treatment effects.