My company has multiple development environments based on multiple features being tested. Instead of everyone maintain and point to each other’s respective feature environment, I am thinking of setting up a whitelisted user based routing setup. For example:
This is easier to do for http requests because haproxy / nginx easily support routing http requests to backends based on the value of say a user_id header.
Things get messy when you want to do this for binary protocols like thrift.
Have written a custom router config in haproxy which I wanted to share.
frontend thriftrouter
bind *:10090
mode tcp
tcp-request inspect-delay 100ms
option tcplog
log global
log-format "%ci:%cp -> %fi:%fp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq captured_user:%{+Q}[capture.req.hdr(0)] req.len:%[capture.req.hdr(1)]"
tcp-request content capture req.payload(52,10) len 10
tcp-request content capture req.len len 10
tcp-request content accept if WAIT_END
acl acl_thrift_call req.payload(2,2) -m bin 0001 # Thrift CALL method
acl acl_magic_field_id req.payload(30,2) -m bin 270f # Magic field number 9999
# Define access control list for each user
acl acl_user_u1 req.payload(0,0),hex -m sub 7573657231 # user1
acl acl_user_u2 req.payload(0,0),hex -m sub 7573657232 # user2
# Route based on the user. No default backend so that one always has to set it
use_backend backend_1 if acl_user_u1 acl_magic_field_id acl_thrift_call
use_backend backend_2 if acl_user_u2 acl_magic_field_id acl_thrift_call
backend backend_1
mode tcp
option tcp-check
server server1 127.0.0.1:9090
backend backend_2
mode tcp
option tcp-check
server server2 107.0.0.1:9091
So now the only missing piece is the java code to generate custom metadata into thrift.
/**
* Injects custom data into the egress thrift call.
*
* @author Gagandeep Singh
*/
@Slf4j
public class CustomInjectorProtocol extends TProtocolDecorator {
private static final short FIELD_ID = 9999; // Magic number - this will be field id of custom data
private final Function<Void, Map<String, String>> function;
public CustomInjectorProtocol(TProtocol protocol, Function<Void, Map<String, String>> function) {
super(protocol);
this.function = function;
}
@Override
public void writeMessageBegin(TMessage tMessage) throws TException {
super.writeMessageBegin(tMessage);
Map<String, String> map;
try {
map = function.apply(null);
log.info("Injecting custom data into egress thrift call");
} catch (Exception e) {
log.error("Error in applying function: {}", e.getMessage());
log.debug("Exception: ", e);
map = new HashMap<>();
}
super.writeFieldBegin(new TField("custom_data", TType.MAP, FIELD_ID));
super.writeMapBegin(new TMap(TType.STRING, TType.STRING, map.size()));
for (Map.Entry<String, String> entry : map.entrySet()) {
super.writeString(entry.getKey());
super.writeString(entry.getValue());
}
super.writeMapEnd();
super.writeFieldEnd();
}
}
Set the user_id when making the thrift call.
TTransport transport = new TSocket("localhost", 10090, 1000000);
transport.open();
TProtocol protocol = new TBinaryProtocol(transport);
TProtocol spanProtocol = new CustomInjectorProtocol(protocol, (v) -> {
Map<String, String> map = Maps.newHashMap();
map.put("userid", "|user1|");
return map;
});
client = new Service.Client(spanProtocol);