First of all, it is very important to comment about the phrase.
In english it is called idiom which is defined as an expression whose meaning is not predictable from the usual meaning of its constituent elements. (Source: dictionary).
In this case the meaning of phrase " you had to carry a bucket".
It suggests that the person could not to do some activity and he/she knew it.
