There are at least a couple of ways to reasonably define this; Nameless has one of them:
(x/y) mod b,
meaning to find an integer c such that
x = (y*c) mod b.
This won't always possible if y and b are not relatively prime. For instance
5/6 mod 12
won't have an answer, because no integer multiple of 6 can ever be = 5 mod 12.
Here's another possible definition.
When you take a mod b, where a and b are positive integers, with b≠0, you are solving this equation for r:
a = b*q + r,
where q is an integer, and
0 ≤ r < b
This is known in mathematics as the "division algorithm," with:
a = dividend
b = divisor (which is not allowed to = 0)
q = quotient
r = remainder
This definition can be extended in a couple of ways -- to negative numbers, and to real numbers, a, b, and r, but always with the proviso that q must be an integer, and that b≠0. For b < 0, the constraint on r becomes:
b < r ≤ 0
Under this definition,
2/7 mod 12 = 2/7
[2/7 = 0*12 + 2/7; 0 ≤ 2/7 < 12]
This is essentially Carlos B's answer. You can also get
5/6 mod 12 = 5/6 [5/6 = 0*12 + 5/6; 0 ≤ 5/6 < 12]
-5/6 mod 12 = 67/6 [-5/6 = -1*12 + 67/6; 0 ≤ 67/6 < 12]
125/6 mod 12 = 53/6 [125/6 = 1*12 + 53/6; 0 ≤ 53/6 < 12]
12 mod (2/7) = 0 [12 = 42*(2/7) + 0; 0 ≤ 0 < 2/7]
12 mod (5/7) = 4/7 [12 = 16*(5/7) + 4/7; 0 ≤ 4/7 < 5/7]
12 mod (-5/7) = -1/7 [12 = -17*(-5/7) + (-1/7); -5/7 < -1/7 ≤ 0]
etc.