Page:Tickle v Giggle for Girls Pty Ltd (No 2).pdf/37

This page has been proofread, but needs to be validated.

selfie would then be reviewed by an AI system provided by a third-party, Kairos, which was designed and used to determine if the person in the selfie appeared to be female or male. The AI system performed what was described as gender detection on user-submitted selfies, purportedly detecting the gender of each face with 94% accuracy. Ms Grover says that she selected that accuracy level as it reduced the chance of women being denied access to the App due to a poor-quality onboarding selfie, but still ensured most men attempting to join were rejected.

102 If the AI system determined that the person appeared female, it would grant them access to the Giggle App. If the potential user was rejected, a pop up would tell the potential user "oops you have not been verified please try again" and they would not receive access to the App or any of its functionality. Giggle would not receive a notification when a potential user was rejected by the AI system, but if their phone number was provided, the potential users' profile could be accessed by those at Giggle through a backend server called Athena. Ms Grover in cross-examination disagreed that the purpose of the Kairos AI was to determine if a person seeking to join the Giggle App was male or female, stating that the purpose of the AI was to "aid us in gatekeeping and creating a female-only space".

103 In addition to the AI screening, Ms Grover, and others running the Giggle App, reviewed the selfies uploaded by users who had been granted access to the app, receiving some kind of daily summary of these users' photographs. She says that this review process was undertaken on a shift basis by herself, her mother and a friend. Ms Grover described the purpose of this additional check as ensuring that men attempting to join the Giggle App had not bypassed the AI screening. Where, in these reviews, a man was identified, he would be blocked from using the Giggle App. On the available evidence, and given Ms Grover's pivotal role, it is more probable than not that, as a practical matter, Ms Grover herself was the one reviewing these images most of the time that the Giggle App was operational, notwithstanding the shift arrangement that she referred to. Again, this could have been clarified by cross-examination, but was not.

104 In her affidavit evidence, Ms Grover says that users who were not granted access by the AI would often email Giggle and Ms Grover could, by using their number, review their profile on Athena. She says Giggle's practice was, on reviewing the applicant's photo, to accept them if they appeared female, reject them if they appeared male, and ask them to submit another photo if their sex was ambiguous. In cross-examination, however, she said that emails received at a


Tickle v Giggle for Girls Pty Ltd (No 2) [2024] FCA 960
30