Like in English, when a quote contains a quote within, the quotation marks alternative. Typically the single quotation marks (「」) are used for the outermost quote, and the innermost has double quotation marks (『』).
Hanako wa "hai" to itta
花子は「はい」と言った
Hanako said "yes."
Tarou wa "Hanako wa 'hai' to itta" to itta
太郎は「花子は『はい』と言った」と言った
Tarou said: "Hanako said: 'yes.'"
Both of which state that 『』are called 二重鉤括弧(にじゅうかぎかっこ) literally double hook brackets. And the normal ones 「」are called 鉤括弧(かぎかっこ) literally hook brackets.
They also state that 『』are typically used for quotes inside of other quotes.
Additionally, the Wikipedia article on Japanese typographical symbols says that they’re used for titles. (Under brackets and quotation marks, second item)