GitRec, The Live Demo

GitRec, the live demo, is developed to demonstrate the usage of Gorse recommender system engine. A user logins in via GitHub OAuth, then repositories starred by this user before are imported to Gorse. Gorse recommends repositories to this user based on starred repositories. When this user saw a recommended repository, he or she can press ❤️ to tell GitRec that he or she like this recommendationn and press ⏯️ to skip current recommendationn.

Design

  • Import new repositories: The trending crawler crawls trending repositories and insert them into Gorse as new items. Since there are huge nuumber of repositories in GitHub, it's impossible add all of them into GitRec. So, only trending repositories are imported.
  • Import user starred repositories: The user starred crawler crawls user starred repositories and insert them into Gorse as new fewdback typed star when a new user signed in.
  • Recommendation and feedbacks: GitRec web service pulls recommendations from Gorse and show to users. When a user press ❤️, a feedback typed like will be inserted to Gorse. When ⏯️ pressed, a feedback typed view will be inserted to Gorse.

Configuration

In GitRec, there are three types of feedbacks: read, star and like. read is the feedback type for read events (a user skip a recommended repository). like is the feedback type for user press ❤️. star is the feedback type for user starred repositories. Since star events doesn't happen in GitRec, it won't be added to click_feedback_types.

# feedback type for positive event
positive_feedback_types = ["star","like"]

# feedback type for click event
click_feedback_types = ["like"]

# feedback type for read event
read_feedback_type = "read"

Other settings are the same as docker/config.toml.

Implemenetation

This project consists of frontend, backend and crawlers. In this section, only these codes interact with Gorse are introduced. Other codes are avaliable in the GitRec repository.

  1. First thing is to wrap Gorse API as a python module. It sends HTTP requests and handle responses using requests.
from collections import namedtuple
from datetime import datetime
from typing import List

import requests

Success = namedtuple("Success", ["RowAffected"])


class GorseException(BaseException):
    def __init__(self, status_code: int, message: str):
        self.status_code = status_code
        self.message = message


class Gorse:
    def __init__(self, entry_point):
        self.entry_point = entry_point

    def insert_feedback(
        self, feedback_type: str, user_id: str, item_id: str
    ) -> Success:
        r = requests.post(
            self.entry_point + "/api/feedback",
            json=[
                {
                    "FeedbackType": feedback_type,
                    "UserId": user_id,
                    "ItemId": item_id,
                    "Timestamp": datetime.now().isoformat(),
                }
            ],
        )
        if r.status_code == 200:
            return r.json()
        raise GorseException(r.status_code, r.text)

    def get_recommend(self, user_id: str, n: int = 1) -> List[str]:
        r = requests.get(self.entry_point + "/api/recommend/%s?n=%d" % (user_id, n))
        if r.status_code == 200:
            return r.json()
        raise GorseException(r.status_code, r.text)

    def insert_feedbacks(self, feedbacks) -> Success:
        r = requests.post(self.entry_point + "/api/feedback", json=feedbacks)
        if r.status_code == 200:
            return r.json()
        raise GorseException(r.status_code, r.text)

    def insert_item(self, item) -> List[str]:
        r = requests.post(self.entry_point + "/api/item", json=item)
        if r.status_code == 200:
            return r.json()
        raise GorseException(r.status_code, r.text)
  1. In the code of trending crawler, insert trending repositories as new items.
if __name__ == "__main__":
    trending_repos = get_trending()
    for trending_repo in trending_repos:
        gorse_client.insert_item(get_repo_info(trending_repo))
  1. In the code of starred repositories crawler, insert user starred repositories as star feedbacks.
@app.task
def pull(token: str):
    g = GraphQLGitHub(token)
    stars = g.get_viewer_starred()
    gorse_client.insert_feedbacks(stars)
  1. In the code of web service backend, pull recommendations from Gorse, insert like and read into Gorse.
@app.route("/api/repo")
def get_repo():
    repo_id = gorse_client.get_recommend(session["user_id"])[0]
    full_name = repo_id.replace(":", "/")
    github_client = Github(github.token["access_token"])
    repo = github_client.get_repo(full_name)
    # ...


@app.route("/api/like/<repo_name>")
def like_repo(repo_name: str):
    try:
        return gorse_client.insert_feedback("like", session["user_id"], repo_name)
    except gorse.GorseException as e:
        return Response(e.message, status=e.status_code)


@app.route("/api/read/<repo_name>")
def read_repo(repo_name: str):
    try:
        return gorse_client.insert_feedback("read", session["user_id"], repo_name)
    except gorse.GorseException as e:
        return Response(e.message, status=e.status_code)