src package

Subpackages

Submodules

src.clause_generator module

class src.clause_generator.ClauseGenerator(args, NSFR, lang, pos_data_loader, mode_declarations, bk_clauses, device, no_xil=False)[source]

Bases: object

clause generator by refinement and beam search :param ilp_problem: :type ilp_problem: .ilp_problem.ILPProblem :param infer_step: number of steps in forward inference :type infer_step: int :param max_depth: max depth of nests of function symbols :type max_depth: int :param max_body_len: max number of atoms in body of clauses :type max_body_len: int

beam_search(C_0, T_beam=7, N_beam=20, N_max=100)[source]

generate clauses by beam-searching from initial clauses Inputs —— C_0 : Set[.logic.Clause]

set of initial clauses

T_beamint: number of steps in beam-searching
N_beamint: size of the beam
N_maxint: maximum number of clauses to be generated

Returns: C – a set of generated clauses
Return type: Set[.logic.Clause]

beam_search_clause(clause, T_beam=7, N_beam=20, N_max=100, th=0.98)[source]

perform beam-searching from a clause Inputs —— clause : Clause

initial clause

T_beamint: number of steps in beam-searching
N_beamint: size of the beam
N_maxint: maximum number of clauses to be generated

Returns: C – a set of generated clauses
Return type: Set[.logic.Clause]

eval_clauses(clauses)[source]

generate(C_0, gen_mode='beam', T_beam=7, N_beam=20, N_max=100)[source]

call clause generation function with or without beam-searching Inputs —— C_0 : Set[.logic.Clause]

a set of initial clauses

gen_modestring: a generation mode ‘beam’ - with beam-searching ‘naive’ - without beam-searching
T_beamint: number of steps in beam-searching
N_beamint: size of the beam
N_maxint: maximum number of clauses to be generated

Returns: C – set of generated clauses
Return type: Set[.logic.Clause]

is_in_beam(B, clause, score)[source]: If score is the same, same predicates => duplication

src.data_clevr module

class src.data_clevr.CLEVRHans(dataset, split, img_size=128, base=None)[source]

Bases: Dataset

CLEVRHans dataset. The implementations is mainly from https://github.com/ml-research/NeSyConceptLearner/blob/main/src/pretrain-slot-attention/data.py.

class src.data_clevr.CLEVRHans_POSITIVE(dataset, split, img_size=128, base=None)[source]

Bases: Dataset

CLEVRHans dataset. The implementations is mainly from https://github.com/ml-research/NeSyConceptLearner/blob/main/src/pretrain-slot-attention/data.py.

src.data_clevr.load_images_and_labels(dataset='clevr-hans3', split='train')[source]: Load image paths and labels for clevr-hans dataset.

src.data_clevr.load_images_and_labels_positive(dataset='clevr-hans0', split='train')[source]: Load image paths and labels for clevr-hans dataset.

src.data_kandinsky module

class src.data_kandinsky.KANDINSKY(dataset, split, img_size=128, small_data=False)[source]

Bases: Dataset

Kandinsky Patterns dataset.

class src.data_kandinsky.KANDINSKY_POSITIVE(dataset, split, img_size=128, small_data=False)[source]

Bases: Dataset

Kandinsky Patterns dataset.

src.data_kandinsky.letterbox(img, new_shape=(640, 640), color=(114, 114, 114), auto=True, scaleFill=False, scaleup=True, stride=32)[source]: A utilitiy function for yolov5 model to make predictions. The implementation is from the yolov5 repository.

src.data_kandinsky.load_image_yolo(path, img_size, stride=32)[source]: Load an image using given path.

src.data_kandinsky.load_images_and_labels(dataset='twopairs', split='train', img_size=128, small_data=False)[source]: Load image paths and labels for kandinsky dataset.

src.data_kandinsky.load_images_and_labels_positive(dataset='twopairs', split='train', img_size=128, small_data=False)[source]: Load image paths and labels for kandinsky dataset.

src.eval_clause_infer module

class src.eval_clause_infer.EvalInferModule(I, m, infer_step, gamma=0.01, device=None, train=False)[source]

Bases: Module

A class of differentiable foward-chaining inference.

forward(x)[source]: In the forward function we accept a Tensor of input data and we must return a Tensor of output data. We can use Modules defined in the constructor as well as arbitrary operators on Tensors.

init_identity_weights(device)[source]

r(x)[source]

training: bool

src.facts_converter module

class src.facts_converter.FactsConverter(lang, perception_module, valuation_module, device=None)[source]

Bases: Module

FactsConverter converts the output fromt the perception module to the valuation vector.

call(pred)[source]

convert(Z, G, B)[source]

convert_i(zs, G)[source]

filter_by_datatype()[source]

forward(Z, G, B)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

get_params()[source]

init_valuation(n, batch_size)[source]

to_vec(term, zs)[source]

training: bool

src.infer module

class src.infer.ClauseFunction(I_i, gamma=0.01)[source]

Bases: Module

A class of the clause function.

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class src.infer.ClauseInferModule(I, infer_step, gamma=0.01, device=None, train=False, m=1, I_bk=None)[source]

Bases: Module

forward(x)[source]: In the forward function we accept a Tensor of input data and we must return a Tensor of output data. We can use Modules defined in the constructor as well as arbitrary operators on Tensors.

r(x)[source]

r_bk(x)[source]

training: bool

class src.infer.InferModule(I, infer_step, gamma=0.01, device=None, train=False, m=1, I_bk=None)[source]

Bases: Module

forward(x)[source]: In the forward function we accept a Tensor of input data and we must return a Tensor of output data. We can use Modules defined in the constructor as well as arbitrary operators on Tensors.

get_params()[source]

r(x)[source]

r_bk(x)[source]

training: bool

src.infer.init_identity_weights(X, device)[source]

src.logic_utils module

src.logic_utils.build_clause_infer_module(clauses, bk_clauses, atoms, lang, device, m=3, infer_step=3, train=False)[source]

src.logic_utils.build_infer_module(clauses, bk_clauses, atoms, lang, device, m=3, infer_step=3, train=False)[source]

src.logic_utils.generate_atoms(lang)[source]

src.logic_utils.generate_bk(lang)[source]

src.logic_utils.get_index_by_predname(pred_str, atoms)[source]

src.logic_utils.get_lang(lark_path, lang_base_path, dataset_type, dataset)[source]

Load the language of first-order logic from files.

Read the language, clauses, background knowledge from files. Atoms are generated from the language.

src.logic_utils.get_searched_clauses(lark_path, lang_base_path, dataset_type, dataset)[source]

Load the language of first-order logic from files.

Read the language, clauses, background knowledge from files. Atoms are generated from the language.

src.logic_utils.parse_clauses(lang, clause_strs)[source]

src.mode_declaration module

class src.mode_declaration.ModeDeclaration(mode_type, recall, pred, mode_terms, ordered=True)[source]

Bases: object

from https://www.cs.ox.ac.uk/activities/programinduction/Aleph/aleph.html p(ModeType, ModeType,…)

Here are some examples of how they appear in a file:

:- mode(1,mem(+number,+list)). :- mode(1,dec(+integer,-integer)). :- mode(1,mult(+integer,+integer,-integer)). :- mode(1,plus(+integer,+integer,-integer)). :- mode(1,(+integer)=(#integer)). :- mode(*,has_car(+train,-car)). Each ModeType is either (a) simple; or (b) structured. A simple ModeType is one of: (a) +T specifying that when a literal with predicate symbol p appears in a hypothesised clause, the corresponding argument should be an “input” variable of type T; (b) -T specifying that the argument is an “output” variable of type T; or (c) #T specifying that it should be a constant of type T. All the examples above have simple modetypes. A structured ModeType is of the form f(..) where f is a function symbol, each argument of which is either a simple or structured ModeType. Here is an example containing a structured ModeType:

To make this more clear, here is an example for the mode declarations for the grandfather task from

above::- modeh(1, grandfather(+human, +human)).:-

modeb(*, parent(-human, +human)).:-
modeb(*, male(+human)). The first mode states that the head of the rule

(and therefore the targetpredicate) will be the atomgrandfather.

Its parameters have to be of the typehuman.

The + annotation says that the rule head needs two variables.

Thesecond mode declaration states theparentatom and declares again

that theparameters have to be of type human.
Here, the + at the second parametertells, that the system is only allowed to introduce the atomparentin the clauseif it already contains a variable of type human.

The−at the first attribute in-troduces a new variable into the clause.

The modes consist of a recall n that states how many versions of the literal are allowed in a rule and an atom with place-markers that state the literal to-gether with annotations on input- and output-variables as well as constants (see[Mug95]). :param recall: The recall number i.e. how many times the declaration can be instanciated :type recall: int :param pred: The predicate. :type pred: Predicate :param mode_terms: Terms for mode declarations. :type mode_terms: ModeTerm

class src.mode_declaration.ModeTerm(mode, dtype)[source]

Bases: object

Terms for mode declarations. It has mode (+, -, #) and data types.

src.mode_declaration.get_mode_declarations(args, lang, obj_num)[source]

src.mode_declaration.get_mode_declarations_clevr(lang, obj_num)[source]

src.mode_declaration.get_mode_declarations_kandinsky(lang, obj_num)[source]

src.neural_utils module

class src.neural_utils.LogisticRegression(input_dim, output_dim=1)[source]

Bases: Module

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class src.neural_utils.MLP(in_channels, out_channels, hidden_dim=256)[source]

Bases: Module

forward(inputs)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

src.nsfr module

class src.nsfr.NSFReasoner(perception_module, facts_converter, infer_module, clause_infer_module, atoms, bk, clauses, train=False)[source]

Bases: Module

The Neuro-Symbolic Forward Reasoner.

Parameters

perception_model (nn.Module) – The perception model.
facts_converter (nn.Module) – The facts converter module.
infer_module (nn.Module) – The differentiable forward-chaining inference module.
atoms (list(atom)) – The set of ground atoms (facts).

atoms_to_text(atoms)[source]

clause_eval(x)[source]

forward(x)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

get_clauses()[source]

get_params()[source]

get_top_atoms(v)[source]

get_valuation_text(valuation)[source]

predict(v, predname)[source]: Extracting a value from the valuation tensor using a given predicate.

predict_multi(v, prednames)[source]

Extracting values from the valuation tensor using given predicates.

prednames = [‘kp1’, ‘kp2’, ‘kp3’]

print_program()[source]: Print asummary of logic programs using continuous weights.

print_valuation_batch(valuation, n=40)[source]

training: bool

src.nsfr_utils module

src.nsfr_utils.denormalize_clevr(imgs)[source]: denormalize clevr images

src.nsfr_utils.denormalize_kandinsky(imgs)[source]: denormalize kandinsky images

src.nsfr_utils.generate_captions(V, atoms, e, th)[source]

src.nsfr_utils.get_clevr_loader(args)[source]

src.nsfr_utils.get_clevr_pos_loader(args)[source]

src.nsfr_utils.get_data_loader(args)[source]

src.nsfr_utils.get_data_pos_loader(args)[source]

src.nsfr_utils.get_kandinsky_loader(args, shuffle=False)[source]

src.nsfr_utils.get_kandinsky_pos_loader(args, shuffle=False)[source]

src.nsfr_utils.get_nsfr_model(args, lang, clauses, atoms, bk, bk_clauses, device, train=False)[source]

src.nsfr_utils.get_nsfr_model_train(args, lang, clauses, atoms, bk, device, m)[source]

src.nsfr_utils.get_prob(v_T, NSFR, args)[source]

if args.dataset_type == ‘kandinsky’:

predicted = NSFR.predict(v=v_T, predname=’kp’)

elif args.dataset_type == ‘clevr’:

if args.dataset == ‘clevr-hans3’:

predicted = NSFR.predict_multi(: v=v_T, prednames=[‘kp1’, ‘kp2’, ‘kp3’])

if args.dataset == ‘clevr-hans7’:

predicted = NSFR.predict_multi(: v=v_T, prednames=[‘kp1’, ‘kp2’, ‘kp3’, ‘kp4’, ‘kp5’, ‘kp6’, ‘kp7’])

src.nsfr_utils.get_prob_by_prednames(v_T, NSFR, prednames)[source]

src.nsfr_utils.save_images_with_captions(imgs, captions, folder, img_id_start, dataset)[source]

src.nsfr_utils.to_plot_images_clevr(imgs)[source]

src.nsfr_utils.to_plot_images_kandinsky(imgs)[source]

src.nsfr_utils.update_initial_clauses(clauses, obj_num)[source]

src.nsfr_utils.update_nsfr_clauses(nsfr, clauses, bk_clauses, device)[source]

src.nsfr_utils.valuation_to_attr_string(v, atoms, e, th=0.5)[source]: Generate string explanations of the scene.

src.nsfr_utils.valuation_to_rel_string(v, atoms, th=0.5)[source]

src.nsfr_utils.valuation_to_string(v, atoms, e, th=0.5)[source]

src.nsfr_utils.valuations_to_string(V, atoms, e, th=0.5)[source]: Generate string explanation of the scenes.

src.optimizer module

class src.optimizer.WeightOptimizer(infer_module, train_idxs, labels, lr=0.01, wd=0.0)[source]

Bases: object

optimizer of clause weights using gradient descent

minibatch(probs, labels)[source]

get minibatch Inputs —— probs : torch.tensor((|train_idxs|, ))

valuation vector of examples each dimension represents each example of the ilp problem

labelstorch.tensor((|train_idxs|, )): label vector of examples each dimension represents each example of the ilp problem

Returns

probs_batch (torch.tensor((batch_size, ))) – valuation vector of examples selected in the minibatch each dimension represents each example of the ilp problem
labels_batch (torch.tensor((batch_size, ))) – label vector of examples selected in the minibatch each dimension represents each example of the ilp problem

optimize_weights(epoch=500)[source]

perform gradient descent to optimize clause weights Inputs —— epoch : int

number of steps in gradient descent

Returns

IM (.infer.InferModule) – infer module that contains optimized weight vectors
loss_list (List[float]) – list of training loss

set_optimizer(params)[source]: set torch optimizer

src.percept module

class src.percept.SlotAttentionPerceptionModule(e, d, device, train=False)[source]

Bases: Module

A perception module using Slot Attention.

Attrs:: e (int): The maximum number of entities. d (int): The dimension of the object-centric vector. device (device): The device where the model and tensors are loaded. train (bool): The flag if the parameters are trained. preprocess (tensor->tensor): Reshape the yolo output into the unified format of the perceptiom module. model: The slot attention model.

forward(imgs)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

load_model()[source]: Load slot attention network.

training: bool

class src.percept.YOLOPerceptionModule(e, d, device, train=False)[source]

Bases: Module

A perception module using YOLO.

Attrs:: e (int): The maximum number of entities. d (int): The dimension of the object-centric vector. device (device): The device where the model and tensors are loaded. train (bool): The flag if the parameters are trained. preprocess (tensor->tensor): Reshape the yolo output into the unified format of the perceptiom module.

forward(imgs)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

load_model(path, device)[source]

pad_result(output)[source]: Padding the result by zeros. (batch, n_obj, 6) -> (batch, n_max_obj, 6)

training: bool

class src.percept.YOLOPreprocess(device, img_size=128)[source]

Bases: Module

A perception module using Slot Attention.

Attrs:: device (device): The device where the model to be loaded. img_size (int): The size of the (resized) image to normalize the xy-coordinates. classes (list(str)): The classes of objects. colors (tensor(int)): The one-hot encodings of the colors (repeated 3 times). shapes (tensor(int)): The one-hot encodings of the shapes (repeated 3 times).

forward(x)[source]

A preprocess funciton for the YOLO model. The format is: [x1, y1, x2, y2, prob, class].

Parameters: x (tensor) – The output of the YOLO model. The format is:
Returns: The preprocessed object-centric representation Z. The format is: [x1, y1, x2, y2, color1, color2, color3, shape1, shape2, shape3, objectness]. x1,x2,y1,y2 are normalized to [0-1]. The probability for each attribute is obtained by copying the probability of the classification of the YOLO model.
Return type: Z (tensor)

training: bool

src.predict_clevr module

src.predict_clevr.get_args()[source]

src.predict_clevr.main()[source]

src.predict_clevr.predict(NSFR, loader, args, device, writer, split='train')[source]

src.predict_kandinsky module

src.predict_kandinsky.get_args()[source]

src.predict_kandinsky.main()[source]

src.predict_kandinsky.predict(NSFR, loader, args, device, writer, th=None, split='train')[source]

src.tensor_encoder module

class src.tensor_encoder.TensorEncoder(lang, facts, clauses, device)[source]

Bases: object

The tensor encoder for differentiable inference.

A class for tensor encoding in differentiable forward-chaining approach.

Parameters

lang (language) – The language of first-order logic.
facts (list(atom)) – The set of ground atoms (facts).
clauses (list(clause)) – The set of clauses (rules).
device (torch.device) – The device to be used.

Attrs:: lang (language): The language of first-order logic. facts (list(atom)): The set of ground atoms (facts). clauses (list(clause)): The set of clauses (rules). G (int): The number of ground atoms. C (int): The number of clauses. L (int): The maximum length of the clauses. S (int): The maximum number of substitutions for body atoms. head_unifier_dic ({(atom, atom) -> List[(var, const)]}): The dictionary to save the computed unification results. fact_index_dic ({atom -> int}): The dictionary that maps an atom to its index.

body_to_tensor(body)[source]

Convert the body atoms into a tensor.

Parameters: body (list(atom)) – The body atoms.
Returns: The tensor representation of the body atoms.
Return type: I_c_b (tensor;(S * L))

build_I_c(clause)[source]

Build index tensor for a given clause.

Parameters: clause (clause) – A clause.
Returns: The index tensor for the given clause (G, S, L).
Return type: I_c (tensor)

build_fact_index_dic()[source]

Build dictionary {fact -> index}

Returns: A dictionary to map the atoms to indices.
Return type: dic ({atom -> int})

build_head_unifier_dic()[source]

Build dictionary {(head, fact) -> unifier}.

Returns: A dictionary to map the pair of ground atoms to their unifier.
Return type: dic ({(atom,atom) -> subtitution})

encode()[source]

Compute the index tensor for the differentiable inference.

Returns I (tensor): The index tensor (G, C, S, L).

facts_to_index(atoms)[source]: Convert given ground atoms into the indices.

generate_subs(body)[source]

Generate substitutions from given body atoms.

Generate the possible substitutions from given list of atoms. If the body contains any variables, then generate the substitutions by enumerating constants that matches the data type. !!! ASSUMPTION: The body has variables that have the same data type

e.g. variables O1(object) and Y(color) cannot appear in one clause !!!

Parameters: body (list(atom)) – The body atoms which may contain existentially quantified variables.
Returns: The list of substitutions of the given body atoms.
Return type: theta_list (list(substitution))

get_fact_index(fact)[source]: Convert a fact to the index in the ordered set of all facts.

get_max_subs_num(clauses)[source]

Compute S (the maximum numebr of substitutions for body atoms) from clauses.

Parameters: clauses (list(clause)) – A set of clauses.
Returns: The maximum number of substitutions for existentially quantified variables in the body atoms.
Return type: S (int)

pad_by_true(x)[source]

Fill the tensor by ones for the clause which has less body atoms than the longest clause.

Parameters: x (tensor) – The tensor.
Returns: The tensor that is padded to the shape of (S, L).
Return type: x_padded (tensor)

src.torch_utils module

src.torch_utils.logsumexp(inputs, dim=None, keepdim=False)[source]

Numerically stable logsumexp. from https://github.com/pytorch/pytorch/issues/2591#issuecomment-364474328 :param inputs: A Variable with any shape. :param dim: An integer. :param keepdim: A boolean.

Returns: Equivalent of log(sum(exp(inputs), dim=dim, keepdim=keepdim)).

src.torch_utils.print_valuation(valuation, atoms, n=40)[source]

Print the valuation tensor.

Print the valuation tensor using given atoms. :param valuation: A valuation tensor. :type valuation: tensor;(B*G) :param atoms: The ground atoms. :type atoms: list(atom)

src.torch_utils.softor(xs, dim=0, gamma=0.01)[source]

The softor function.

Parameters

xs (tensor or list(tensor)) – The input tensor.
dim (int) – The dimension to be removed.
(float (gamma) – The smooth parameter for logsumexp.

Returns

The result of taking or along dim.

Return type

log_sum_exp (tensor)

src.torch_utils.weight_sum(W_l, H)[source]

src.train module

src.train.discretise_NSFR(NSFR, args, device)[source]

src.train.get_args()[source]

src.train.main(n)[source]

src.train.predict(NSFR, loader, args, device, th=None, split='train')[source]

src.train.train_nsfr(args, NSFR, optimizer, train_loader, val_loader, test_loader, device, writer, rtpt)[source]

src.train_cnn module

src.train_cnn.compute_acc(outputs, targets)[source]

src.train_cnn.get_args()[source]

src.train_cnn.main(n)[source]

src.train_cnn.predict(net, predict_net, loader, device, th=None)[source]

src.train_cnn.run(net, predict_net, loader, optimizer, criterion, writer, args, device, train=False, epoch=0, rtpt=None, max_obj_num=4)[source]

src.train_cnn_clevr module

src.train_cnn_clevr.compute_acc(outputs, targets)[source]

src.train_cnn_clevr.get_args()[source]

src.train_cnn_clevr.main(n)[source]

src.train_cnn_clevr.predict(net, predict_net, loader, device, th=None)[source]

src.train_cnn_clevr.run(net, predict_net, loader, optimizer, criterion, writer, args, device, train=False, epoch=0, rtpt=None, max_obj_num=4)[source]

src.train_sa_mlp_clevr module

src.train_sa_mlp_clevr.compute_acc(outputs, targets)[source]

src.train_sa_mlp_clevr.get_args()[source]

src.train_sa_mlp_clevr.main(n)[source]

src.train_sa_mlp_clevr.predict(net, predict_net, loader, device, th=None)[source]

src.train_sa_mlp_clevr.run(net, predict_net, loader, optimizer, criterion, writer, args, device, train=False, epoch=0, rtpt=None, max_obj_num=4)[source]

src.train_yolo_mlp module

src.train_yolo_mlp.compute_acc(outputs, targets)[source]

src.train_yolo_mlp.get_args()[source]

src.train_yolo_mlp.main(n)[source]

src.train_yolo_mlp.predict(net, predict_net, loader, device, th=None)[source]

src.train_yolo_mlp.run(net, predict_net, loader, optimizer, criterion, writer, args, device, train=False, epoch=0, rtpt=None, max_obj_num=4)[source]

src.valuation module

class src.valuation.SlotAttentionValuationModule(lang, device, pretrained=True)[source]

Bases: Module

A module to call valuation functions. Attrs:

lang (language): The language. device (device): The device. layers (list(nn.Module)): The list of valuation functions. vfs (dic(str->nn.Module)): The dictionaty that maps a predicate name to the corresponding valuation function. attrs (dic(term->tensor)): The dictionary that maps an attribute term to the corresponding one-hot encoding. dataset (str): The dataset.

forward(zs, atom)[source]

Convert the object-centric representation to a valuation tensor.

Parameters

zs (tensor) – The object-centric representaion (the output of the YOLO model).
atom (atom) – The target atom to compute its proability.

Returns

A batch of the probabilities of the target atom.

ground_to_tensor(term, zs)[source]

Ground terms into tensor representations.

Parameters

term (term) – The term to be grounded.
zs (tensor) – The object-centric representation.

init_valuation_functions(device, pretrained)[source]

Parameters

device (device) – The device.
pretrained (bool) – The flag if the neural predicates are pretrained or not.

Retunrs:: layers (list(nn.Module)): The list of valuation functions. vfs (dic(str->nn.Module)): The dictionaty that maps a predicate name to the corresponding valuation function.

term_to_onehot(term, batch_size)[source]

Ground terms into tensor representations.

Parameters

term (term) – The term to be grounded.
zs (tensor) – The object-centric representation.

Returns

The tensor representation of the input term.

to_onehot_batch(i, length, batch_size)[source]: Compute the one-hot encoding that is expanded to the batch size.

training: bool

class src.valuation.YOLOValuationModule(lang, device, dataset)[source]

Bases: Module

A module to call valuation functions. Attrs:

lang (language): The language. device (device): The device. layers (list(nn.Module)): The list of valuation functions. vfs (dic(str->nn.Module)): The dictionaty that maps a predicate name to the corresponding valuation function. attrs (dic(term->tensor)): The dictionary that maps an attribute term to the corresponding one-hot encoding. dataset (str): The dataset.

forward(zs, atom)[source]

Convert the object-centric representation to a valuation tensor.

Parameters

zs (tensor) – The object-centric representaion (the output of the YOLO model).
atom (atom) – The target atom to compute its proability.

Returns

A batch of the probabilities of the target atom.

ground_to_tensor(term, zs)[source]

Ground terms into tensor representations.

Parameters

term (term) – The term to be grounded.
zs (tensor) – The object-centric representation.

Returns

The tensor representation of the input term.

init_attr_encodings(device)[source]

Encode color and shape into one-hot encoding.

Parameters: device (device) – The device.
Returns: The dictionary that maps an attribute term to the corresponding one-hot encoding.
Return type: attrs (dic(term->tensor))

init_valuation_functions(device, dataset=None)[source]

Parameters

device (device) – The device.
dataset (str) – The dataset.

Retunrs:: layers (list(nn.Module)): The list of valuation functions. vfs (dic(str->nn.Module)): The dictionaty that maps a predicate name to the corresponding valuation function.

training: bool

src.valuation_func module

class src.valuation_func.SlotAttentionColorValuationFunction(device)[source]

Bases: Module

The function v_color.

forward(z, a)[source]

Parameters

z (tensor) –
2-d tensor (B * D), the object-centric representation. obj_prob + coords + shape + size + material + color [objectness, x, y, z, sphere, cube, cylinder, large, small, rubber,

metal, cyan, blue, yellow, purple, red, green, gray, brown]
a (tensor) – The one-hot tensor that is expanded to the batch size.

Returns

A batch of probabilities.

training: bool

class src.valuation_func.SlotAttentionFrontValuationFunction(device)[source]

Bases: Module

The function v_infront.

forward(z_1, z_2)[source]

Parameters

z (tensor) –

2-d tensor (B * D), the object-centric representation. obj_prob + coords + shape + size + material + color [objectness, x, y, z, sphere, cube, cylinder, large, small, rubber,

metal, cyan, blue, yellow, purple, red, green, gray, brown]

Returns

A batch of probabilities.

training: bool

class src.valuation_func.SlotAttentionInValuationFunction(device)[source]

Bases: Module

The function v_in.

forward(z, x)[source]

Parameters

z (tensor) –
2-d tensor (B * D), the object-centric representation. obj_prob + coords + shape + size + material + color [objectness, x, y, z, sphere, cube, cylinder, large, small, rubber,

metal, cyan, blue, yellow, purple, red, green, gray, brown]
x (none) – A dummy argment to represent the input constant.

Returns

A batch of probabilities.

training: bool

class src.valuation_func.SlotAttentionLeftSideValuationFunction(device)[source]

Bases: Module

The function v_leftside.

forward(z)[source]

Parameters

z (tensor) –

2-d tensor (B * D), the object-centric representation. obj_prob + coords + shape + size + material + color [objectness, x, y, z, sphere, cube, cylinder, large, small, rubber,

metal, cyan, blue, yellow, purple, red, green, gray, brown]

Returns

A batch of probabilities.

training: bool

class src.valuation_func.SlotAttentionMaterialValuationFunction(device)[source]

Bases: Module

The function v_material.

forward(z, a)[source]

Parameters

z (tensor) –
2-d tensor (B * D), the object-centric representation. obj_prob + coords + shape + size + material + color [objectness, x, y, z, sphere, cube, cylinder, large, small, rubber,

metal, cyan, blue, yellow, purple, red, green, gray, brown]
a (tensor) – The one-hot tensor that is expanded to the batch size.

Returns

A batch of probabilities.

training: bool

class src.valuation_func.SlotAttentionRightSideValuationFunction(device)[source]

Bases: Module

The function v_rightside.

forward(z)[source]

Parameters

z (tensor) –

2-d tensor (B * D), the object-centric representation. obj_prob + coords + shape + size + material + color [objectness, x, y, z, sphere, cube, cylinder, large, small, rubber,

metal, cyan, blue, yellow, purple, red, green, gray, brown]

Returns

A batch of probabilities.

training: bool

class src.valuation_func.SlotAttentionShapeValuationFunction(device)[source]

Bases: Module

The function v_shape.

forward(z, a)[source]

Parameters

z (tensor) –
2-d tensor (B * D), the object-centric representation. obj_prob + coords + shape + size + material + color [objectness, x, y, z, sphere, cube, cylinder, large, small, rubber,

metal, cyan, blue, yellow, purple, red, green, gray, brown]
a (tensor) – The one-hot tensor that is expanded to the batch size.

Returns

A batch of probabilities.

training: bool

class src.valuation_func.SlotAttentionSizeValuationFunction(device)[source]

Bases: Module

The function v_size.

forward(z, a)[source]

Parameters

z (tensor) –
2-d tensor (B * D), the object-centric representation. obj_prob + coords + shape + size + material + color [objectness, x, y, z, sphere, cube, cylinder, large, small, rubber,

metal, cyan, blue, yellow, purple, red, green, gray, brown]
a (tensor) – The one-hot tensor that is expanded to the batch size.

Returns

A batch of probabilities.

training: bool

class src.valuation_func.YOLOClosebyValuationFunction(device)[source]

Bases: Module

The function v_closeby.

forward(z_1, z_2)[source]

Parameters

z_1 (tensor) –
2-d tensor (B * D), the object-centric representation. [x1, y1, x2, y2, color1, color2, color3,

shape1, shape2, shape3, objectness]
z_2 (tensor) –
2-d tensor (B * D), the object-centric representation. [x1, y1, x2, y2, color1, color2, color3,

shape1, shape2, shape3, objectness]

Returns

A batch of probabilities.

to_center(z)[source]

training: bool

class src.valuation_func.YOLOColorValuationFunction[source]

Bases: Module

The function v_color.

forward(z, a)[source]

Parameters

z (tensor) –
2-d tensor B * d of object-centric representation. [x1, y1, x2, y2, color1, color2, color3,

shape1, shape2, shape3, objectness]
a (tensor) – The one-hot tensor that is expanded to the batch size.

Returns

A batch of probabilities.

training: bool

class src.valuation_func.YOLOInValuationFunction[source]

Bases: Module

The function v_in.

forward(z, x)[source]

Parameters

z (tensor) –
2-d tensor (B * D), the object-centric representation. [x1, y1, x2, y2, color1, color2, color3,

shape1, shape2, shape3, objectness]
x (none) – A dummy argment to represent the input constant.

Returns

A batch of probabilities.

training: bool

class src.valuation_func.YOLOOnlineValuationFunction(device)[source]

Bases: Module

The function v_online.

forward(z_1, z_2, z_3, z_4, z_5)[source]

The function to compute the probability of the online predicate.

The closed form of the linear regression is computed. The error value is fed into the 1-d logistic regression function.

Parameters

z_i (tensor) –

2-d tensor (B * D), the object-centric representation. [x1, y1, x2, y2, color1, color2, color3,

shape1, shape2, shape3, objectness]

Returns

A batch of probabilities.

to_center_x(z)[source]

to_center_y(z)[source]

training: bool

class src.valuation_func.YOLOShapeValuationFunction[source]

Bases: Module

The function v_shape.

forward(z, a)[source]

Parameters

z (tensor) –
2-d tensor (B * D), the object-centric representation. [x1, y1, x2, y2, color1, color2, color3,

shape1, shape2, shape3, objectness]
a (tensor) – The one-hot tensor that is expanded to the batch size.

Returns

A batch of probabilities.

training: bool

src package

Subpackages

Submodules

src.clause_generator module

src.data_clevr module

src.data_kandinsky module

src.eval_clause_infer module

src.facts_converter module

src.infer module

src.logic_utils module

src.mode_declaration module

src.neural_utils module

src.nsfr module

src.nsfr_utils module

src.optimizer module

src.percept module

src.predict_clevr module

src.predict_kandinsky module

src.refinement module

src.tensor_encoder module

src.torch_utils module

src.train module

src.train_cnn module

src.train_cnn_clevr module

src.train_sa_mlp_clevr module

src.train_yolo_mlp module

src.valuation module

src.valuation_func module

Module contents