dw软件入门教程/郑州百度推广seo

自定义节流

有些时候为了对用户的访问频率进行限制和防止爬虫，需要在规定的时间中对用户访问的次数进行限制

 

下面自定义一个用户每分钟只能访问3次，代码如下：

from rest_framework.throttling import BaseThrottle
import time
VISIT_RECORD = {}   #保存访问记录class VisitThrottle(BaseThrottle):'''60s内只能访问3次'''def __init__(self):self.history = None   #初始化访问记录def allow_request(self,request,view):#获取用户ip (get_ident)remote_addr = self.get_ident(request)ctime = time.time()#如果当前IP不在访问记录里面，就添加到记录if remote_addr not in VISIT_RECORD:VISIT_RECORD[remote_addr] = [ctime,]     #键值对的形式保存return True    #True表示可以访问#获取当前ip的历史访问记录history = VISIT_RECORD.get(remote_addr)#初始化访问记录self.history = history#如果有历史访问记录，并且最早一次的访问记录离当前时间超过60s，就删除最早的那个访问记录，#只要为True，就一直循环删除最早的一次访问记录while history and history[-1] < ctime - 60:history.pop()#如果访问记录不超过三次，就把当前的访问记录插到第一个位置（pop删除最后一个）if len(history) < 3:history.insert(0,ctime)return Truedef wait(self):'''还需要等多久才能访问'''ctime = time.time()return 60 - (ctime - self.history[-1])

 

上面的大码就是当用户第一次访问的时候把它的IP地址和当前访问的时间添加到字典  VISIT_RECORD 中， 循环取出最先添加的时间判断其时间有没有过了60s，如果过了则将其删掉，然后在看列表的长度是否小于3次，如果小于3次证明，是可以访问的的。wait表示的是还需要等待多久可以继续访问

我们在使用的时候，只需在类视图中简单在类属性  throttle_classes = [ 节流控制的类] 即可

 

下面我们在用户登陆的时候来简单的测试下，代码代码如下：

class AuthView(APIView):"""用于用户登录认证"""authentication_classes = []permission_classes = []throttle_classes = [VisitThrottle,] # 进行访问次数的限制def post(self,request,*args,**kwargs):ret = {'code':1000,'msg':None}try:user = request._request.POST.get('username')pwd = request._request.POST.get('password')obj = models.UserInfo.objects.filter(username=user,password=pwd).first()if not obj:ret['code'] = 1001ret['msg'] = "用户名或密码错误"# 为登录用户创建tokentoken = md5(user)# 存在就更新，不存在就创建models.UserToken.objects.update_or_create(user=obj,defaults={'token':token})ret['token'] = tokenexcept Exception as e:print(e)ret['code'] = 1002ret['msg'] = '请求异常'return JsonResponse(ret)

 我们连续访问3次登陆测试的结果如下

 

节流源码分析

 源码入口 dispatch 代码如下

    def dispatch(self, request, *args, **kwargs):"""`.dispatch()` is pretty much the same as Django's regular dispatch,but with extra hooks for startup, finalize, and exception handling."""self.args = argsself.kwargs = kwargs# 对原生的request进行封装request = self.initialize_request(request, *args, **kwargs)self.request = requestself.headers = self.default_response_headers  # deprecate?try:self.initial(request, *args, **kwargs)# Get the appropriate handler methodif request.method.lower() in self.http_method_names:handler = getattr(self, request.method.lower(),self.http_method_not_allowed)else:handler = self.http_method_not_allowedresponse = handler(request, *args, **kwargs)except Exception as exc:response = self.handle_exception(exc)self.response = self.finalize_response(request, response, *args, **kwargs)return self.response

 

 执行认证  self.initial 代码如下

    def initial(self, request, *args, **kwargs):"""Runs anything that needs to occur prior to calling the method handler."""self.format_kwarg = self.get_format_suffix(**kwargs)# Perform content negotiation and store the accepted info on the requestneg = self.perform_content_negotiation(request)request.accepted_renderer, request.accepted_media_type = neg# Determine the API version, if versioning is in use.version, scheme = self.determine_version(request, *args, **kwargs)request.version, request.versioning_scheme = version, scheme# Ensure that the incoming request is permitted# 实现认证
        self.perform_authentication(request)# 权限判断
        self.check_permissions(request)# 访问频率控制self.check_throttles(request)

 

 

 访问频率控制self.check_throttles(request)代码如下：

    def check_throttles(self, request):"""Check if request should be throttled.Raises an appropriate exception if the request is throttled."""for throttle in self.get_throttles():if not throttle.allow_request(request, self):self.throttled(request, throttle.wait())

 

在上面的源码中我们可以知道，如果没有通过会返回 false 执行 self.throttled(request, throttle.wait()),抛出异常，代码如下

 def throttled(self, request, wait):"""If request is throttled, determine what kind of exception to raise."""raise exceptions.Throttled(wait)

 

如果 返回True表示可以继续访问，让我们来继续追踪  self.get_throttles() 代码如下：

    def get_throttles(self):"""Instantiates and returns the list of throttles that this view uses."""# 返回访问频率控制的列表return [throttle() for throttle in self.throttle_classes]

 

通过上面的源码我们知道为什么我们在需要进行节流限制的接口中设置类属性 throttle_class = [ 节流限制的类 ] ，还有就是在我们节流限制的类中为什么要重写  

allow_request(request, self) 和 wait 方法  

进行全局节流的配置

 上面是我们自己定义的节流控制，走我们自己定义的类属性     throttle_class = [ 节流限制的类 ]  如果我们想使用父类的从而实现全局的配置又该如何去实现呢

 

父类中的 thtottle_classes   设置如下

 

 

 所以我们只需在配置文件中的   REST_FRAMEWORK  添加  DEFAULT_THROTTLE_CLASSES 的路径即可，配置如下

REST_FRAMEWORK = {# 全局使用的认证类"DEFAULT_AUTHENTICATION_CLASSES":['api.utils.auth.FirstAuthtication','api.utils.auth.Authtication', ],"UNAUTHENTICATED_USER":lambda :"匿名用户","UNAUTHENTICATED_TOKEN":None,"DEFAULT_PERMISSION_CLASSES":['api.utils.permission.SVIPPermission'],# 默认的权限认证"DEFAULT_THROTTLE_CLASSES":["api.utils.throttle.VisitThrottle"], # 进行节流的限制
}

 

根据上面节流限制类路径的定义，我们在应用 api 下的utils目录下创建 throttle.py 把代码如下

from rest_framework.throttling import BaseThrottle,SimpleRateThrottleimport time
VISIT_RECORD = {}
class VisitThrottle(BaseThrottle):def __init__(self):self.history = Nonedef allow_request(self,request,view):# 1. 获取用户IPremote_addr = self.get_ident(request)ctime = time.time()if remote_addr not in VISIT_RECORD:VISIT_RECORD[remote_addr] = [ctime,]return Truehistory = VISIT_RECORD.get(remote_addr)self.history = historywhile history and history[-1] < ctime - 60:history.pop()if len(history) < 3:history.insert(0,ctime)return True# return True    # 表示可以继续访问# return False # 表示访问频率太高，被限制def wait(self):# 还需要等多少秒才能访问ctime = time.time()return 60 - (ctime - self.history[-1])

 

我们的视图类中就不需要再添加节流的类属性的配置了，就可以实现节流的控制。简单的代码如下

class AuthView(APIView):"""用于用户登录认证"""authentication_classes = []permission_classes = []def post(self,request,*args,**kwargs):pass

 

内置的控制频率类

 上面是写的自定义节流，drf内置了很多节流的类，用起来比较方便。

（1）BaseThrottle

• 自己要写allow_request和wait方法
• get_ident就是获取ip

 

class BaseThrottle(object):"""Rate throttling of requests."""def allow_request(self, request, view):"""Return `True` if the request should be allowed, `False` otherwise."""raise NotImplementedError('.allow_request() must be overridden')def get_ident(self, request):"""Identify the machine making the request by parsing HTTP_X_FORWARDED_FORif present and number of proxies is > 0. If not use all ofHTTP_X_FORWARDED_FOR if it is available, if not use REMOTE_ADDR."""xff = request.META.get('HTTP_X_FORWARDED_FOR')remote_addr = request.META.get('REMOTE_ADDR')num_proxies = api_settings.NUM_PROXIESif num_proxies is not None:if num_proxies == 0 or xff is None:return remote_addraddrs = xff.split(',')client_addr = addrs[-min(num_proxies, len(addrs))]return client_addr.strip()return ''.join(xff.split()) if xff else remote_addrdef wait(self):"""Optionally, return a recommended number of seconds to wait beforethe next request."""return None

 

我们上面的例子就是通过继承  BaseThrottle 来实现的，相对较麻烦一点

（2）SimpleRateThrottle

class SimpleRateThrottle(BaseThrottle):"""A simple cache implementation, that only requires `.get_cache_key()`to be overridden.The rate (requests / seconds) is set by a `rate` attribute on the Viewclass.  The attribute is a string of the form 'number_of_requests/period'.Period should be one of: ('s', 'sec', 'm', 'min', 'h', 'hour', 'd', 'day')Previous request information used for throttling is stored in the cache."""cache = default_cachetimer = time.timecache_format = 'throttle_%(scope)s_%(ident)s'scope = None   #这个值自定义，写什么都可以THROTTLE_RATES = api_settings.DEFAULT_THROTTLE_RATESdef __init__(self):if not getattr(self, 'rate', None):self.rate = self.get_rate()self.num_requests, self.duration = self.parse_rate(self.rate)def get_cache_key(self, request, view):"""Should return a unique cache-key which can be used for throttling.Must be overridden.May return `None` if the request should not be throttled."""raise NotImplementedError('.get_cache_key() must be overridden')def get_rate(self):"""Determine the string representation of the allowed request rate."""if not getattr(self, 'scope', None):msg = ("You must set either `.scope` or `.rate` for '%s' throttle" %self.__class__.__name__)raise ImproperlyConfigured(msg)try:return self.THROTTLE_RATES[self.scope]except KeyError:msg = "No default throttle rate set for '%s' scope" % self.scoperaise ImproperlyConfigured(msg)def parse_rate(self, rate):"""Given the request rate string, return a two tuple of:<allowed number of requests>, <period of time in seconds>"""if rate is None:return (None, None)num, period = rate.split('/')num_requests = int(num)duration = {'s': 1, 'm': 60, 'h': 3600, 'd': 86400}[period[0]]return (num_requests, duration)def allow_request(self, request, view):"""Implement the check to see if the request should be throttled.On success calls `throttle_success`.On failure calls `throttle_failure`."""if self.rate is None:return Trueself.key = self.get_cache_key(request, view)if self.key is None:return Trueself.history = self.cache.get(self.key, [])self.now = self.timer()# Drop any requests from the history which have now passed the# throttle durationwhile self.history and self.history[-1] <= self.now - self.duration:self.history.pop()if len(self.history) >= self.num_requests:return self.throttle_failure()return self.throttle_success()def throttle_success(self):"""Inserts the current request's timestamp along with the keyinto the cache."""self.history.insert(0, self.now)self.cache.set(self.key, self.history, self.duration)return Truedef throttle_failure(self):"""Called when a request to the API has failed due to throttling."""return Falsedef wait(self):"""Returns the recommended next request time in seconds."""if self.history:remaining_duration = self.duration - (self.now - self.history[-1])else:remaining_duration = self.durationavailable_requests = self.num_requests - len(self.history) + 1if available_requests <= 0:return Nonereturn remaining_duration / float(available_requests)

 让我们来继续追踪以下其内部的源码，我们来看一下 函数

 

    def allow_request(self, request, view):"""Implement the check to see if the request should be throttled.On success calls `throttle_success`.On failure calls `throttle_failure`."""if self.rate is None:return True# 获取一个用于缓存的键来记录限流的self.key = self.get_cache_key(request, view)if self.key is None:return Trueself.history = self.cache.get(self.key, [])self.now = self.timer()# Drop any requests from the history which have now passed the# throttle durationwhile self.history and self.history[-1] <= self.now - self.duration:self.history.pop()if len(self.history) >= self.num_requests:return self.throttle_failure()return self.throttle_success()

 

 

我们可以看到   它的内部首先判断 self.rate 是否为None ，我们不知道其是什么不放追踪以下看看

 if not getattr(self, 'rate', None):self.rate = self.get_rate()self.num_requests, self.duration = self.parse_rate(self.rate)

 

get_rate 的代码如下

    def get_rate(self):"""Determine the string representation of the allowed request rate."""if not getattr(self, 'scope', None):msg = ("You must set either `.scope` or `.rate` for '%s' throttle" %self.__class__.__name__)raise ImproperlyConfigured(msg)try:return self.THROTTLE_RATES[self.scope]except KeyError:msg = "No default throttle rate set for '%s' scope" % self.scoperaise ImproperlyConfigured(msg)

 

 

我们看到 如果 没有定义self.scope这个属性的话，直接会抛出异常，所以在这里如果我们要是继承  SimpleRateThrottle 类来实现限流的话，必须定义类属性 scope ，这个是需要我们自己写的

 

让我们继续追踪代码下面的   self.THROTTLE_RATES[self.scope]

 

    THROTTLE_RATES = api_settings.DEFAULT_THROTTLE_RATES

 

我们可以看到它是从配置文件中读取  DEFAULT_THROTTLE_CLASSES  可以看到它是从配置文件中根据键 self.scrope去进行读取的

 

让我们来继续追踪  allow_request 中的   self.key = self.get_cache_key(request, view)  其代码如下：

    def get_cache_key(self, request, view):"""Should return a unique cache-key which can be used for throttling.Must be overridden.May return `None` if the request should not be throttled."""raise NotImplementedError('.get_cache_key() must be overridden')

 

在上面的源码中，我们发现其抛出了一个异常， 这个是用来 获取一个用于缓存的键来记录限流的，所以这个方法我们要重写，重写的代码如下

from rest_framework.throttling import BaseThrottle,SimpleRateThrottle
class VisitThrottle(SimpleRateThrottle):scope = "NBA"def get_cache_key(self, request, view): return self.get_ident(request) # 默认根据IP作为键 class UserThrottle(SimpleRateThrottle): scope = "NBAUser" def get_cache_key(self, request, view): return request.user.username # 登陆后用户名作为键

 

 

下缅的流程就和我们自定义的限流的逻辑一样了，早这里就不做陈述了

 

继承 SimpleRateThrottle的写法如下

我们可以通过继承SimpleRateThrottle类，来实现节流，会更加的简单，因为SimpleRateThrottle里面都帮我们写好了，

 

from rest_framework.throttling import SimpleRateThrottleclass VisitThrottle(SimpleRateThrottle):'''匿名用户60s只能访问三次（根据ip）'''scope = 'NBA'   #这里面的值，自己随便定义，settings里面根据这个值配置Ratedef get_cache_key(self, request, view):#通过ip限制节流return self.get_ident(request)class UserThrottle(SimpleRateThrottle):'''登录用户60s可以访问10次'''scope = 'NBAUser'    #这里面的值，自己随便定义，settings里面根据这个值配置Ratedef get_cache_key(self, request, view):return request.user.username

（2）settings.py

 

#全局
REST_FRAMEWORK = {#节流"DEFAULT_THROTTLE_CLASSES":['API.utils.throttle.UserThrottle'],   #全局配置，登录用户节流限制（10/m）"DEFAULT_THROTTLE_RATES":{'NBA':'3/m',         #没登录用户3/m，NBA就是scope定义的值'NBAUser':'10/m',    #登录用户10/m，NBAUser就是scope定义的值
    }
}

 

（3）views.py

局部配置方法

class AuthView(APIView):..    .# 默认的节流是登录用户（10/m）,AuthView不需要登录，这里用匿名用户的节流（3/m）throttle_classes = [VisitThrottle,].

 

说明：

• API.utils.throttle.UserThrottle   这个是全局配置（根据ip限制，10/m）
• DEFAULT_THROTTLE_RATES      --->>>设置访问频率的
• throttle_classes = [VisitThrottle,]     --->>>局部配置（不适用settings里面默认的全局配置）

 

总结

基本使用

• 创建类，继承BaseThrottle, 实现：allow_request ，wait  
• 创建类，继承SimpleRateThrottle,   实现：  get_cache_key， scope='NBA'      (配置文件中的key)    

全局

   #节流"DEFAULT_THROTTLE_CLASSES":['API.utils.throttle.UserThrottle'],   #全局配置，登录用户节流限制（10/m）"DEFAULT_THROTTLE_RATES":{'NBA':'3/m',         #没登录用户3/m，NBA就是scope定义的值'NBAUser':'10/m',    #登录用户10/m，NBAUser就是scope定义的值
    }
}

局部

throttle_classes = [VisitThrottle,]